An object size measurement method, device, computer equipment and storage medium

Abstract

The present disclosure provides an object size measurement method and device, computer equipment and storage medium, wherein the method comprises: acquiring a target image containing an object to be measured; determining a plurality of first candidate lines in the target image based on an auxiliary region of the target image; wherein the auxiliary region is a preset region for displaying the object to be measured; determining the endpoint coordinates of the first candidate lines, and determining the frame line of the object to be measured in the target image based on the endpoint coordinates; and taking the size information of the frame line as the size information of the object to be measured.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to an object size measurement method and device, a computer device and a storage medium. BACKGROUND

[0002] With the development of technology, more and more users replace traditional door locks with smart door locks. When replacing the door lock, the size of the guide piece in the traditional door lock needs to be determined to determine the model of the smart door lock.

[0003] In related technologies, the size of the guide piece is generally measured manually with the help of a ruler. This method needs to use professional measuring tools, which is inconvenient for measurement. In addition, the measurement result may have a large error due to manual measurement by the user. SUMMARY

[0004] The present disclosure provides at least an object size measurement method and device, a computer device and a storage medium.

[0005] In a first aspect, the present disclosure provides an object size measurement method, comprising:

[0006] obtaining a target image containing an object to be measured;

[0007] determining a plurality of first candidate lines in the target image based on an auxiliary region of the target image; wherein the auxiliary region is a preset region for displaying the object to be measured;

[0008] determining end point coordinates of the first candidate lines, and determining a frame line of the object to be measured in the target image based on the end point coordinates;

[0009] taking the size information of the frame line as the size information of the object to be measured.

[0010] In the above method, the target image of the object to be measured can be photographed, and a plurality of first candidate lines in the target image can be automatically identified. Then, the frame line of the object to be measured is determined from the first candidate lines. The size information of the frame line is taken as the size information of the object to be measured. In this way, the frame line of the object to be measured can be accurately selected by photographing the image, and the pixel size of the frame line of the object to be measured can be accurately calculated.

[0011] In a possible implementation, the determining of the plurality of first candidate lines in the target image based on the auxiliary region of the target image comprises:

[0012] determining a plurality of second candidate lines in the target image based on a pre-trained neural network;

[0013] Determine a first candidate line from the second candidate lines based on an auxiliary region of the target image.

[0014] In a possible implementation, the pre-trained neural network comprises a sampling network and a straight line verification network.

[0015] The pre-trained neural network is configured to determine the second candidate lines in the target image by:

[0016] The sampling network is configured to perform feature extraction on the target image to determine a feature map corresponding to the target image, and predict an intersection in the target image based on the sampling network.

[0017] Determine a third candidate line based on the intersection.

[0018] The third candidate line, the feature map, and the straight line verification network are configured to determine the second candidate lines in the target image.

[0019] The sampling network can accurately determine the third candidate line in the target image, and the straight line verification network can accurately filter out the second candidate lines that can be edges of objects in the target image.

[0020] In a possible implementation, the pre-trained neural network is configured to determine the second candidate lines in the target image by:

[0021] For each third candidate line from the third candidate lines, determine a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line in the feature map.

[0022] Input the feature vectors corresponding to the third candidate lines into the straight line verification network one by one to obtain the second candidate lines in the target image.

[0023] In a possible implementation, the inputting of the feature vectors corresponding to the third candidate lines into the straight line verification network one by one to obtain the second candidate lines in the target image comprises:

[0024] Input the feature vector corresponding to the third candidate line into the straight line verification network to obtain a confidence score corresponding to the third candidate line, wherein the confidence score represents a probability that the third candidate line is an edge of an object.

[0025] The third candidate line that satisfies a preset condition is determined as the second candidate line.

[0026] Using the confidence level as a filtering condition, multiple third candidate lines that can serve as object edges in the target image can be accurately filtered out, and the ability of the straight line verification network to filter the second candidate lines can be easily adjusted.

[0027] In one possible implementation, the method further includes performing the following steps to train the neural network:

[0028] A sample image carrying first annotation information and second annotation information is obtained, wherein the first annotation information is used to represent the intersection of object bounding boxes in the sample image; and the second annotation information is used to represent the object bounding boxes in the sample image.

[0029] The sample image is input into the sampling network to obtain the feature map corresponding to the sample image and the prediction intersection point corresponding to the sample image;

[0030] Based on the predicted intersection points, multiple prediction candidate lines are determined, and feature vectors corresponding to the multiple prediction candidate lines are determined based on the feature map.

[0031] Based on the feature vectors corresponding to the line verification network and the multiple prediction candidate lines, multiple prediction lines are determined.

[0032] A first loss value is determined based on the predicted intersection point and the first annotation information, and a second loss value is determined based on the predicted line and the second annotation information;

[0033] The neural network is trained based on the first loss value and the second loss value.

[0034] Training the neural network can improve its ability to predict the intersection points, thereby improving its ability to determine the second candidate line.

[0035] In one possible implementation, determining multiple first candidate lines from among the multiple second candidate lines based on the auxiliary region of the target image includes:

[0036] Based on the side length of the auxiliary region, determine the length range used to filter the first candidate line;

[0037] Based on the length range and the preset angle range, the first candidate line among the plurality of second candidate lines is determined.

[0038] In one possible implementation, determining the bounding box of the object to be measured in the target image based on the endpoint coordinates includes:

[0039] For each of the multiple first candidate lines, the starting point coordinates and ending point coordinates of the first candidate line are determined based on the preset straight line direction and the endpoint coordinates.

[0040] determine two first target candidate lines with the same starting point and different straight directions;

[0041] determine two second target candidate lines with the same ending point and different starting points respectively being the ending points of the two first target candidate lines;

[0042] determine the bounding box of the object to be measured based on the first target candidate lines and the second target candidate lines.

[0043] The first target candidate lines and the second target candidate lines that can form a rectangular bounding box can be accurately screened out through the matching of the starting points and the ending points of the first candidate lines.

[0044] In a possible implementation, the method further includes:

[0045] In a case where there are multiple detection bounding boxes determined based on the first target candidate lines and the second target candidate lines, the bounding box of the object to be measured is determined based on the areas of the detection bounding boxes.

[0046] In a possible implementation, after the bounding box of the object to be measured in the target image is determined based on the endpoint coordinates, the method further includes:

[0047] In a case where the pattern formed by the bounding box of the object to be measured is not a rectangle, the bounding box of the object to be measured is corrected.

[0048] After the bounding box of the object to be measured is corrected, the actual size of the bounding box of the object to be measured can be more accurately calculated.

[0049] In a second aspect, the embodiments of the present disclosure further provide an object size measurement device, including:

[0050] an acquisition module configured to acquire a target image containing an object to be measured;

[0051] a first determination module configured to determine a plurality of first candidate lines in the target image based on an auxiliary region of the target image, wherein the auxiliary region is a preset region for displaying the object to be measured;

[0052] a second determination module configured to determine endpoint coordinates of the first candidate lines and determine a bounding box of the object to be measured in the target image based on the endpoint coordinates;

[0053] a size measurement module configured to take size information of the bounding box as size information of the object to be measured.

[0054] In a possible implementation, the first determining module, when determining the plurality of first candidate lines in the target image based on the auxiliary region of the target image, is configured to:

[0055] determine the plurality of second candidate lines in the target image based on the pre-trained neural network;

[0056] determine a plurality of first candidate lines in the plurality of second candidate lines based on the auxiliary region of the target image.

[0057] In a possible implementation, the pre-trained neural network comprises a sampling network and a straight line verification network.

[0058] The first determining module, when determining the plurality of second candidate lines in the target image based on the pre-trained neural network, is configured to:

[0059] perform feature extraction on the target image based on the sampling network to determine a feature map corresponding to the target image, and predict an intersection in the target image based on the sampling network;

[0060] determine a plurality of third candidate lines based on the intersection;

[0061] determine the plurality of second candidate lines in the target image based on the plurality of third candidate lines, the feature map, and the straight line verification network.

[0062] In a possible implementation, the first determining module, when determining the plurality of second candidate lines in the target image based on the plurality of third candidate lines, the feature map, and the straight line verification network, is configured to:

[0063] for each third candidate line in the plurality of third candidate lines, determine a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line in the feature map;

[0064] input the feature vectors corresponding to the plurality of third candidate lines into the straight line verification network one by one to obtain the plurality of second candidate lines in the target image.

[0065] In a possible implementation, the first determining module, when inputting the feature vectors corresponding to the plurality of third candidate lines into the straight line verification network one by one to obtain the plurality of second candidate lines in the target image, is configured to:

[0066] input the feature vector corresponding to the third candidate line into the straight line verification network to obtain a confidence score corresponding to the third candidate line, wherein the confidence score represents a probability that the third candidate line is an object edge;

[0067] The third candidate line satisfying the preset condition is taken as the second candidate line.

[0068] In a possible implementation, the first determining module is further configured to train the neural network by performing the following steps:

[0069] obtain a sample image carrying first annotation information and second annotation information, wherein the first annotation information is used to represent intersection points of object bounding lines in the sample image, and the second annotation information is used to represent the object bounding lines in the sample image;

[0070] input the sample image into the sampling network to obtain a feature map corresponding to the sample image and a predicted intersection point corresponding to the sample image;

[0071] determine a plurality of predicted candidate lines based on the predicted intersection point, and determine a feature vector corresponding to each of the plurality of predicted candidate lines based on the feature map;

[0072] determine a plurality of predicted lines based on the straight line verification network and the feature vector corresponding to each of the plurality of predicted candidate lines;

[0073] determine a first loss value based on the predicted intersection point and the first annotation information, and determine a second loss value based on the predicted lines and the second annotation information;

[0074] train the neural network based on the first loss value and the second loss value.

[0075] In a possible implementation, the first determining module, when determining a plurality of first candidate lines from the plurality of second candidate lines based on the auxiliary region of the target image, is configured to:

[0076] determine a length range for screening the first candidate lines based on a side length of the auxiliary region;

[0077] determine the first candidate lines from the plurality of second candidate lines based on the length range and a preset angle range.

[0078] In a possible implementation, the second determining module, when determining the bounding line of the object to be measured in the target image based on the end point coordinates, is configured to:

[0079] for each of the plurality of first candidate lines, determine a start point coordinate and an end point coordinate of the first candidate line based on a preset straight line direction and the end point coordinates;

[0080] determine two first target candidate lines with the same start point and different straight line directions;

[0081] determine two second target candidate lines respectively starting from the end points of the two first target candidate lines and ending at the same point;

[0082] determine the bounding line of the object to be measured based on the first target candidate line and the second target candidate line.

[0083] In a possible implementation, the second determining module is further configured to:

[0084] In a case where there are multiple detection bounding boxes determined based on the first target candidate line and the second target candidate line, determine the bounding line of the object to be measured based on the area of each detection bounding box.

[0085] In a possible implementation, after determining the bounding line of the object to be measured in the target image based on the end point coordinates, the second determining module is further configured to:

[0086] In a case where the pattern formed by the bounding line of the object to be measured is not a rectangle, correct the bounding line of the object to be measured.

[0087] In a third aspect, the embodiments of the present disclosure further provide a computer device, including a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the first aspect or any possible implementation of the first aspect.

[0088] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the first aspect or any possible implementation of the first aspect.

[0089] The effects of the above object size measuring device, computer device and computer readable storage medium are described in the above object size measuring method, and will not be repeated here.

[0090] In order to make the above objectives, characteristics and advantages of the present disclosure more apparent and easy to understand, the following preferred embodiments are specifically described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0091] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly introduced as follows. The drawings incorporated into the specification and form a part of the specification, which show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0092] Figure 1 A flow chart of a method for measuring the size of an object is shown according to an embodiment of the present disclosure;

[0093] Figure 2 A flow chart of a method for determining a second candidate line is shown according to an embodiment of the present disclosure;

[0094] Figure 3 An architecture diagram of a sampling network is shown according to an embodiment of the present disclosure;

[0095] Figure 4 A flow chart of a method for training a neural network is shown according to an embodiment of the present disclosure;

[0096] Figure 5 A schematic diagram of an auxiliary region is shown according to an embodiment of the present disclosure;

[0097] Figure 6 A schematic diagram of determining a straight line constituting a guide piece line frame is shown according to an embodiment of the present disclosure;

[0098] Figure 7 A schematic diagram of angle screening is shown according to an embodiment of the present disclosure;

[0099] Figure 8 A schematic diagram of determining the start point and end point of a second candidate line is shown according to an embodiment of the present disclosure;

[0100] Figure 9 A schematic diagram of determining a second target candidate line is shown according to an embodiment of the present disclosure;

[0101] Figure 10 A flow chart of a method for detecting guide piece size information is shown according to an embodiment of the present disclosure;

[0102] Figure 11 An architecture diagram of an object size measurement device is shown according to an embodiment of the present disclosure;

[0103] Figure 12A structural diagram of a computer device 1200 is shown. DETAILED DESCRIPTION

[0104] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will be combined with the accompanying drawings to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The components of the embodiments of the present disclosure generally described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0105] With the development of science and technology, more and more users begin to replace traditional door locks with smart door locks. When replacing the door lock, the size of the guide piece in the traditional door lock needs to be determined to determine the model of the smart door lock.

[0106] In the related art, the size of the guide piece is generally measured manually with the help of a ruler. This method needs to use professional measuring tools, which is inconvenient for measurement, and the measurement result may be greatly inaccurate due to manual measurement by the user.

[0107] Based on the above research, the present disclosure provides a method and device for measuring the size of an object, a computer device, and a storage medium. The method can capture a target image of an object to be measured, automatically identify a plurality of first candidate lines in the target image, and then determine a frame line of the object to be measured from the first candidate lines. The size information of the frame line is used as the size information of the object to be measured. This method can accurately select the frame line of the object to be measured by capturing an image and accurately calculate the pixel size of the frame line of the object to be measured.

[0108] The defects of the above solutions are the result of the inventors' practice and careful research, so the discovery process of the above problems and the solutions proposed by the present disclosure to solve the above problems should be the contribution of the inventors to the present disclosure.

[0109] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0110] For the convenience of understanding the present embodiment, first, a kind of object size measurement method disclosed in the present embodiment is introduced in detail, the execution subject of the object size measurement method provided by the present embodiment is generally computer equipment with certain computing power, which includes, for example: terminal equipment or server or other processing equipment, terminal equipment can be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc.In some possible implementation ways, the object size measurement method can be realized by the way that processor calls computer readable instructions stored in memory.

[0111] It should be noted that the "line" (including "straight line", "prediction line", "candidate line") described in the present disclosure is a line segment, or can be understood as a straight line with length, and the "intersection point" is the intersection point of the "line".

[0112] Referring to Figure 1 The flow chart of the object size measurement method provided by the present embodiment is shown, and the method includes steps 101-104, wherein:

[0113] Step 101, obtaining a target image containing an object to be measured;

[0114] Step 102, determining a plurality of first candidate lines in the target image based on an auxiliary region of the target image; wherein the auxiliary region is a preset region for displaying the object to be measured;

[0115] Step 103, determining the endpoint coordinates of the first candidate lines, and determining the frame line of the object to be measured in the target image based on the endpoint coordinates;

[0116] Step 104, taking the size information of the frame line as the size information of the object to be measured.

[0117] The following is a detailed description of the above steps.

[0118] For step 101,

[0119] The object to be measured can be an object with straight edge lines, which can be a door lock guide piece for example; the target image is an image containing the object to be measured.

[0120] In one possible application scenario, the image of the object to be measured can be taken by an image acquisition device to obtain the target image.

[0121] For step 102,

[0122] In a possible implementation, in determining the first candidate lines in the target image based on the auxiliary region of the target image, the second candidate lines in the target image can be determined based on the pre-trained neural network first, and then the first candidate lines in the second candidate lines can be determined based on the auxiliary region of the target image.

[0123] In a possible implementation, the pre-trained neural network can include a sampling network and a straight line verification network. In determining the second candidate lines in the target image based on the pre-trained neural network, the second candidate lines in the target image can be determined by the method as shown in Figure 2 The method includes the following steps:

[0124] Step 201, performing feature extraction on the target image based on the sampling network to determine a feature map corresponding to the target image, and predicting intersection points in the target image based on the sampling network.

[0125] Step 202, determining third candidate lines based on the intersection points.

[0126] Step 203, determining the second candidate lines in the target image based on the third candidate lines, the feature map, and the straight line verification network.

[0127] For step 201,

[0128] In a possible implementation, as shown in Figure 3 The sampling network can be composed of two HourGlass modules, each of which includes a down-sampling module and an up-sampling module, and the output of each HourGlass module is connected to the input through a skip connection.

[0129] The skip connection can be understood as integrating the input information of each HourGlass module into the output information. Through this connection mode, global features and local features can be extracted at the same time, so that the information of the obtained feature map is more comprehensive.

[0130] Here, the intersection points are line segment intersection points or straight line intersection points, which can be understood as the endpoints of the object edge lines. When predicting the intersection points in the target image, the sampling network can predict the position information of the intersection points in the target image. The position information can be represented by position coordinates, for example.

[0131] For step 202,

[0132] In a possible implementation, when the plurality of third candidate lines are determined based on the intersection points, each two of the intersection points can be combined to form a third candidate line.

[0133] For step 203,

[0134] In a possible implementation, after the third candidate lines are determined, the second candidate lines in the target image that can be edges of objects can be determined based on the third candidate lines, the feature map, and the straight line verification network.

[0135] The straight line verification network may, for example, be a multi-layer neural network (MLP), and the structure of the straight line verification network may, for example, be a plurality of times of superposition of linear layers and activation functions (Relu). Specifically, the structure of the straight line verification network may, for example, be that a first activation function is superposed on a first linear layer at a bottom layer, a second linear layer is superposed on the first activation function, a second activation function is superposed on the second linear layer, and a third linear layer is superposed on the second activation function, and a third activation function is superposed on the third linear layer. The first, second, and third activation functions and linear layers are only used to distinguish different layers, and any activation function and linear layer may, for example, be the same as any other activation function and linear layer.

[0136] In a possible implementation, when the second candidate lines in the target image are determined based on the third candidate lines, the feature map, and the straight line verification network, for each third candidate line of the third candidate lines, a feature vector corresponding to the third candidate line may, for example, be determined based on feature values corresponding to points on the third candidate line in the feature map, and the feature vectors corresponding to the third candidate lines may, for example, be input into the straight line verification network one by one to obtain the second candidate lines in the target image.

[0137] For example, for any third candidate line, when a feature vector corresponding to the third candidate line is determined based on feature values corresponding to points on the third candidate line in the feature map, a spacing length between adjacent feature points may, for example, be determined based on a length of the third candidate line and a preset number of feature points, a position of each feature point on the third candidate line may, for example, be determined based on the spacing length, feature information at a position point corresponding to the feature point in the feature map may, for example, be determined based on the position of the feature point, and the feature vector of the third candidate line may, for example, be determined based on the feature information corresponding to the feature points on the third candidate line.

[0138] In a possible implementation, when the feature information of the feature point at the corresponding position on the feature map is determined based on the position of the feature point, the feature information at the corresponding position on the feature map can be directly obtained.

[0139] The feature information corresponding to the feature point can be the value of the feature point on each channel. When the feature vector of the third candidate line is determined based on the feature information corresponding to the feature point on the third candidate line, the values of the feature points on each channel can be concatenated to obtain the feature vector of the third candidate line, and the order of concatenation can be, for example, the position order of the feature points on the third candidate line.

[0140] In a possible implementation, when the feature vectors corresponding to the obtained plurality of third candidate lines are input into the straight line verification network one by one to obtain a plurality of second candidate lines in the target image, the feature vector corresponding to the third candidate line can be input into the straight line verification network to obtain a confidence degree corresponding to the third candidate line; and the third candidate line that satisfies a preset condition can be taken as the second candidate line, where the confidence degree represents the probability that the third candidate line is an object edge, and the preset condition can be, for example, that the confidence degree is higher than a preset confidence degree.

[0141] Alternatively, in another possible implementation, the third candidate lines can be screened according to a preset condition inside the straight line verification network. Specifically, after the straight line verification network determines the confidence degrees corresponding to the third candidate lines, the confidence degree of each third candidate line can be compared with the preset condition to screen and output the second candidate line that satisfies the preset condition.

[0142] The second candidate line screened in this way has high accuracy and ensures that the second candidate line is an edge of an object. It should be noted that the second candidate line is not necessarily an edge of the object to be measured.

[0143] In a possible implementation, the neural network can be trained by the following method, as shown in FIG. Figure 4 The method can include the following steps:

[0144] In step 401, a sample image carrying first annotation information and second annotation information is obtained, where the first annotation information is used to represent the intersection of the object frame line in the sample image, and the second annotation information is used to represent the object frame line in the sample image.

[0145] In step 402, the sample image is input into the sampling network to obtain the feature map corresponding to the sample image and the predicted intersection point corresponding to the sample image.

[0146] Step 403, determining a plurality of predicted candidate lines based on the predicted intersection points, and determining feature vectors corresponding to the plurality of predicted candidate lines based on the feature map;

[0147] Step 404, determining a plurality of predicted lines based on the straight line verification network and the feature vectors corresponding to the plurality of predicted candidate lines respectively;

[0148] Step 405, determining a first loss value based on the predicted intersection points and the first annotation information, and determining a second loss value based on the predicted lines and the second annotation information;

[0149] Step 406, training the neural network based on the first loss value and the second loss value.

[0150] In a possible implementation, when determining the first loss value based on the predicted intersection points and the first annotation information, if the first annotation information is the coordinates of the intersection points in the target image, the first loss value can be determined by calculating the distance between the first annotation information and the corresponding predicted intersection points respectively, and then by calculating the average, variance or the like of the distance.

[0151] In a possible implementation, when determining the second loss value based on the predicted lines and the second annotation information, if the second annotation information is the end point coordinates of the straight line in the target image, the midpoint coordinates between the two end points in the second annotation information can be calculated first, and then the second loss value can be determined based on the distance between the midpoint coordinates of the predicted line and the midpoint coordinates of the second annotation information.

[0152] In a possible implementation, when training the neural network based on the first loss value and the second loss value, the first loss value and the second loss value can be weighted and summed, the sum result can be taken as the loss value in the current training process, and the network parameters of the neural network can be adjusted based on the loss value.

[0153] Alternatively, when training the neural network, the sampling network can be trained based on the first loss value and the second loss value, and the straight line verification network can be trained based on the second loss value.

[0154] In a possible application scenario, after the second candidate line is determined, the first candidate line with a length and an angle that are likely to be the frame line of the object to be measured needs to be screened based on the auxiliary region.

[0155] Here, the auxiliary region can be understood as a region required to display the object to be measured when the target image is captured. For example, the auxiliary region can be a region that is required to display the object to be measured when the target image is captured. Figure 5As shown, the solid-lined box represents the target image, and the dashed-lined box represents the auxiliary area. In one possible implementation, a prompt can be added above the auxiliary area to remind the user to place the object to be measured within the auxiliary area for display. Auxiliary lines can also be added within the auxiliary area to allow the user to photograph the object to be measured from a more suitable angle.

[0156] In practical applications, if the object to be measured is a rectangular object, then to measure the size of the object, the included angle between any two adjacent sides of the frame of the object to be measured should be a right angle. Therefore, the first candidate line can be selected from the second candidate line by setting an angle range. At the same time, there may be some interference lines that are too long in the target image. To avoid detection errors caused by the interference lines, the first candidate line can be selected by setting a length range.

[0157] For example, such as Figure 6 As shown, the auxiliary area includes some straight lines that make up the door frame, straight lines that make up the guide plate, and some other second candidate lines. To measure the dimensions of the guide plate EFHG in the door lock, it is necessary to determine the straight lines EF, FH, HG, and GE that make up the guide plate frame. Therefore, AE, AF, AH, EH, and AG can be eliminated by setting an angle range, and AC and BD can be eliminated by setting a length range, thus obtaining the first candidate lines EF, FH, HG, and GE.

[0158] In one possible implementation, when determining multiple first candidate lines from among the multiple second candidate lines based on an auxiliary region of the target image, the length range for filtering the first candidate lines can be determined first based on the side length of the auxiliary region; the first candidate lines from among the multiple second candidate lines can then be determined based on the length range and a preset angle range. Here, all lengths can be pixel lengths.

[0159] In one possible implementation, when determining the length range for filtering the first candidate line based on the side length of the auxiliary region, since the object to be measured is theoretically located inside the auxiliary region, and the first candidate line is a straight line that may be the edge of the object to be measured, theoretically the first candidate line does not exceed the auxiliary region. For example, the maximum value of the length range can be set to k times the length of the auxiliary region, where k is a positive number less than 1.

[0160] In another possible implementation, if the length and width of the auxiliary region are not equal, the length range can be set based on the shorter of the length and width of the auxiliary region. Alternatively, the second candidate line can be divided into horizontal and vertical lines based on a preset angle range, and then the first candidate line can be filtered based on the length ranges set for the length and width of the region respectively.

[0161] In one possible implementation, when dividing the second candidate line into horizontal and vertical lines based on the preset angle range, the preset angle range may include a horizontal angle range and a vertical angle range. The horizontal angle range may be defined as the acute angle between the second candidate line and the horizontal line not exceeding m degrees, and the vertical angle range may be defined as the acute angle between the second candidate line and the vertical line not exceeding n degrees, where m and n are positive numbers less than 90 degrees. The second candidate line within the horizontal angle range is a horizontal line, and the second candidate line within the vertical angle range is a vertical line. Then, the second candidate lines that do not belong to the angle range can be filtered out.

[0162] For example, such as Figure 7 As shown, the horizontal angle range can be defined as the acute angle between the second candidate line and the horizontal line not exceeding 10 degrees, and the vertical angle range can be defined as the acute angle between the second candidate line and the vertical line not exceeding 10 degrees. If the acute angle between line AB and the horizontal line is within the horizontal angle range, then line AB is a horizontal line. If the angle between line CD and the vertical line is within the vertical angle range, then line CD is a vertical line. Line EF does not belong to either the horizontal angle range or the vertical angle range, so it can be filtered out.

[0163] In one possible implementation, after determining the horizontal and vertical lines based on the above method, the length ranges can be set based on either the length or width of the region. For example, the maximum value of the vertical line's length range can be set to no more than *a* times the width of the auxiliary region, and the maximum value of the horizontal line's length range can be set to no more than *b* times the length of the auxiliary region, where *a* and *b* are positive numbers.

[0164] For example, if the length of the auxiliary area is 300 pixels and the width is 200 pixels, and the maximum value of the length range of the vertical line does not exceed 0.9 times the width of the auxiliary area, and the maximum value of the length range of the horizontal line does not exceed 0.8 times the length of the auxiliary area, then vertical lines with a length of not less than 270 pixels and horizontal lines with a length of not less than 160 pixels will be filtered out.

[0165] In practical applications, if the object to be measured is not completely located within the auxiliary area in the captured target image, that is, part of the object to be measured is located outside the auxiliary area, then in this case, only the size information of the part of the object to be measured located within the auxiliary area can be measured, or the area of ​​the object to be measured cannot be measured.

[0166] In a possible application scenario, to ensure that the object to be measured is completely located in the auxiliary area when the size information is detected, after the target image is acquired, the position information of the object to be measured in the target image can be detected first, and then whether the object to be measured is completely located in the auxiliary area is detected based on the position information. If yes, the subsequent steps of step 101 are executed. If no, prompt information can be output to the user to prompt the user to re-shoot the target image.

[0167] Here, it should be noted that when the position information of the object to be measured in the target image is detected first, the edge of the object to be measured can be detected, and the position information can be determined based on the position of the edge. Here, the edge of the object to be measured detected can be rough edge information.

[0168] For step 103,

[0169] In a possible implementation, after the first candidate line is determined, the end point coordinates of the first candidate line need to be determined, and the frame line of the object to be measured in the target image is determined based on the end point coordinates. Specifically, the following steps can be used:

[0170] Step 1: For each first candidate line in the plurality of first candidate lines, the start point coordinates and the end point coordinates of the first candidate line are determined based on the preset straight line direction and the end point coordinates.

[0171] Step 2: Determine two first target candidate lines with the same start point and different straight line directions.

[0172] Step 3: Determine two second target candidate lines with the same end point and different straight line directions.

[0173] Step 4: Determine the frame line of the object to be measured based on the first target candidate line and the second target candidate line.

[0174] Here, it should be noted that the same start point or the same end point not only includes that the start point coordinates or the end point coordinates are completely the same, but also includes that the distance between the start point coordinates or the end point coordinates is less than a preset difference value. The preset difference value should be set to a small number. The preset difference value is set to avoid errors in screening the first target candidate line and the second target candidate line due to the deviation of the intersection point.

[0175] In a possible implementation, when the end point coordinates of the second candidate line are determined, a pixel coordinate system can be established in the target image. For example, the upper left corner of the target image can be taken as the coordinate origin to establish the pixel coordinate system. Based on the pixel coordinate system, the end point coordinates of the second candidate line can be determined.

[0176] After determining the endpoint coordinates of each second candidate line, the start and end points of each second candidate line can be determined based on these endpoint coordinates. For example, the endpoint closer to the origin in the second candidate line can be set as the start point, and the endpoint farther from the origin can be set as the end point, and the endpoint coordinates can be determined.

[0177] For example, such as Figure 8 As shown, the x-axis is the horizontal axis, the y-axis is the vertical axis, points A and C, which are closer to the origin, are the starting points, and points B and D, which are farther from the origin, are the ending points.

[0178] Alternatively, in another possible implementation, after determining the endpoint coordinates of each second candidate line, the starting point coordinates and ending point coordinates of each second candidate line can be determined based on a preset straight line direction.

[0179] Specifically, the preset straight line direction may include a preset horizontal line direction and a preset vertical line direction. The starting point and ending point of the horizontal line in the second candidate line are determined based on the preset horizontal line direction, and the starting point and ending point of the vertical line in the second candidate line are determined based on the preset vertical line direction.

[0180] For example, if the direction of the horizontal line is from left to right, then the starting point of the horizontal line in the second candidate line should be the leftmost endpoint, and the ending point of the horizontal line should be the rightmost endpoint; if the direction of the vertical line is from top to bottom, then the starting point of the vertical line in the second candidate line should be the topmost endpoint, and the ending point of the vertical line should be the bottommost endpoint.

[0181] In one possible implementation, when determining two first target candidate lines with the same starting point but different straight line directions, the starting point coordinates of any second candidate line can be compared one by one with the starting point coordinates of the other second candidate lines. The resulting first target candidate lines are the two adjacent frame lines of the object to be measured.

[0182] In one possible implementation, when determining the second target candidate line based on the first target candidate line, for example, such as Figure 9 As shown, if the starting coordinates of lines AB and AC are the same, then lines AB and AC are the first target candidate lines. If the ending coordinates of line AB are the same as the starting coordinates of line BE, and the distance between the ending coordinates of line AC and the starting coordinates of line DE is less than the preset deviation, then lines BE and DE are the second target candidate lines.

[0183] In actual applications, after the length screening and the angle screening, there can be multiple second candidate lines that do not belong to the frame line of the object to be measured. Then, based on the multiple second candidate lines, multiple first target candidate lines and multiple second target candidate lines are determined. Each two first target candidate lines and each two second target candidate lines are combined into a detection frame, and multiple detection frames that do not belong to the object to be measured are obtained. The detection frames have shapes similar to the detection frame of the object to be measured, but have different areas from the detection frame of the object to be measured.

[0184] Therefore, in a possible implementation, when the frame line of the object to be measured is determined, if there are multiple detection frames determined based on the first target candidate lines and the second target candidate lines, the frame line of the object to be measured can be determined based on the areas of the detection frames. For example, the frame line that encloses the largest area can be taken as the frame line of the object to be measured.

[0185] In actual applications, the detection frame usually is not a rectangle due to errors in each link, and the integral method can be used to calculate the area of the detection frame.

[0186] In a possible implementation, if the detection frame is not a closed figure, the two endpoints of the broken part can be taken as the endpoints of the frame line to calculate the area enclosed by the frame line, and the largest or smallest area can be taken as the final area enclosed by the frame line. Correspondingly, the endpoints that enclose the largest or smallest area can be taken as the final frame line of the object to be measured. For example, as shown in FIG. 6, the areas of the quadrilateral ACEB and the quadrilateral ADEB are calculated respectively, and the area of the quadrilateral ACEB that has the largest area is taken as the final area enclosed by the frame line. Figure 9

[0187] In another possible implementation, when the frame line of the object to be measured is determined, if there are multiple detection frames determined based on the first target candidate lines and the second target candidate lines, the frame line of the object to be measured can be determined based on the aspect ratios of the various models of the object to be measured that are stored.

[0188] For example, the aspect ratio of each detection frame can be calculated first, the calculated aspect ratio is matched with the stored aspect ratios respectively, and the detection frame that is successfully matched is determined as the frame line of the object to be measured.

[0189] If there are multiple detection frames that are successfully matched, the detection frame that is matched with the aspect ratio of the model most widely used in the market can be taken as the frame line of the object to be measured, or the detection frame that has the largest area can be taken as the frame line of the object to be measured.

[0190] For step 104,​

[0191] In a possible implementation, after determining the frame line of the object to be measured in the target image based on the endpoint coordinates, if it is detected that the pattern formed by the frame line of the object to be measured is not a rectangle, the frame line of the object to be measured is corrected.

[0192] In a possible implementation, whether the frame line is a rectangle can be determined by detecting whether the angle between each two adjacent sides of the frame line of the object to be measured is a right angle, and if the frame line is not a rectangle, the frame line is corrected.

[0193] In a possible implementation, the size information of the frame line can be a pixel size, and the actual information of the frame line can be determined based on the depth information, where the actual information includes the actual length of each straight line constituting the frame line.

[0194] In another possible implementation, the aspect ratio of the frame line is determined and output based on the pixel size of the frame line, and the model of the frame line is determined based on the aspect ratio, so as to determine the actual information of the frame line.

[0195] In the following, the object size measurement method will be introduced by taking a guide sheet as an example, in combination with specific implementations. Referring to FIG. 1, a target image of a guide sheet is captured, and then a candidate straight line (i.e., the second candidate line) is determined from the target image. Figure 10 As shown in FIG. 1, a flowchart of a method for detecting size information of a guide sheet is provided in the embodiment of the present disclosure.

[0196] First, a target image with a guide sheet is captured, and then a candidate straight line (i.e., the second candidate line) is determined from the target image. Then, length screening and angle screening are performed based on the coordinates of the auxiliary region, where the length screening and the angle screening can be performed in any order. After the screening, a vertical candidate line and a horizontal candidate line (i.e., the first candidate line) are obtained, and then the vertical candidate line and the horizontal candidate line are matched and combined to obtain a plurality of possible guide sheet frame lines (through step 103). Then, the guide sheet frame lines are verified (through step 103, for example, the verification can be performed by area), and finally the size of the guide sheet is output.

[0197] The object size measurement method provided in the embodiment of the present disclosure can capture a target image of an object to be measured, and automatically identify a plurality of first candidate lines in the target image, and then determine a frame line of the object to be measured from the first candidate lines. The size information of the frame line is taken as the size information of the object to be measured. In this way, the frame line of the object to be measured can be accurately screened by capturing an image, and the pixel size of the frame line of the object to be measured can be accurately calculated.

[0198] Those skilled in the art can understand that the sequence of writing each step in the above method of the specific embodiment does not mean a strict execution sequence and does not constitute any limitation on the implementation process. The specific execution sequence of each step should be determined by its function and possible internal logic.

[0199] Based on the same inventive concept, the object size measurement device corresponding to the object size measurement method is also provided in the embodiments of the disclosure. Since the device in the embodiments of the disclosure solves the problem by the similar principle as the above-mentioned object size measurement method of the embodiments of the disclosure, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0200] Referring to Figure 11 Fig. 1 shows an architecture schematic diagram of an object size measurement device provided by the embodiments of the disclosure. The device comprises an acquisition module 1101, a first determination module 1102, a second determination module 1103, and a size measurement module 1104; wherein,

[0201] The acquisition module 1101 is configured to acquire a target image containing an object to be measured.

[0202] The first determination module 1102 is configured to determine a plurality of first candidate lines in the target image based on an auxiliary region of the target image; wherein the auxiliary region is a preset region for displaying the object to be measured.

[0203] The second determination module 1103 is configured to determine the end point coordinates of the first candidate lines, and determine a frame line of the object to be measured in the target image based on the end point coordinates.

[0204] The size measurement module 1104 is configured to take the size information of the frame line as the size information of the object to be measured.

[0205] In a possible implementation, when determining the plurality of first candidate lines in the target image based on the auxiliary region of the target image, the first determination module 1102 is configured to:

[0206] determine a plurality of second candidate lines in the target image based on a pre-trained neural network;

[0207] determine a plurality of first candidate lines in the plurality of second candidate lines based on the auxiliary region of the target image.

[0208] In a possible implementation, the pre-trained neural network comprises a sampling network and a straight line verification network.

[0209] When determining the plurality of second candidate lines in the target image based on the pre-trained neural network, the first determination module 1102 is configured to:

[0210] perform feature extraction on the target image based on the sampling network, determine a feature map corresponding to the target image, and predict an intersection point in the target image based on the sampling network;

[0211] determine a plurality of third candidate lines based on the intersection point;

[0212] determine a plurality of second candidate lines in the target image based on the plurality of third candidate lines, the feature map, and the straight line verification network.

[0213] In a possible implementation, the first determining module 1102, when determining the plurality of second candidate lines in the target image based on the plurality of third candidate lines, the feature map, and the straight line verification network, is configured to:

[0214] for each third candidate line in the plurality of third candidate lines, determine a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line in the feature map;

[0215] input the feature vectors corresponding to the plurality of third candidate lines respectively into the straight line verification network one by one to obtain the plurality of second candidate lines in the target image.

[0216] In a possible implementation, the first determining module 1102, when inputting the feature vectors corresponding to the plurality of third candidate lines respectively into the straight line verification network one by one to obtain the plurality of second candidate lines in the target image, is configured to:

[0217] input the feature vector corresponding to the third candidate line into the straight line verification network to obtain a confidence degree corresponding to the third candidate line respectively; wherein the confidence degree represents a probability that the third candidate line is an object edge;

[0218] take the third candidate line whose confidence degree meets a preset condition as the second candidate line.

[0219] In a possible implementation, the first determining module 1102 is further configured to perform the following steps to train the neural network:

[0220] obtain a sample image carrying first annotation information and second annotation information, wherein the first annotation information is used to represent an intersection point of an object frame line in the sample image; and the second annotation information is used to represent the object frame line in the sample image;

[0221] input the sample image into the sampling network to obtain a feature map corresponding to the sample image and a predicted intersection point corresponding to the sample image;

[0222] determine a plurality of predicted candidate lines based on the predicted intersection point, and determine a feature vector corresponding to each of the plurality of predicted candidate lines based on the feature map;

[0223] determine a plurality of predicted lines based on the straight line verification network and the feature vector corresponding to each of the plurality of predicted candidate lines;

[0224] determine a first loss value based on the predicted intersection point and the first annotation information, and determine a second loss value based on the predicted lines and the second annotation information;

[0225] train the neural network based on the first loss value and the second loss value.

[0226] In a possible implementation, the first determining module 1102, when determining a plurality of first candidate lines from the plurality of second candidate lines based on the auxiliary region of the target image, is configured to:

[0227] determine a length range for screening the first candidate lines based on a side length of the auxiliary region;

[0228] determine the first candidate lines from the plurality of second candidate lines based on the length range and a preset angle range.

[0229] In a possible implementation, the second determining module 1103, when determining the bounding line of the object to be measured in the target image based on the endpoint coordinates, is configured to:

[0230] for each of the plurality of first candidate lines, determine a start point coordinate and an end point coordinate of the first candidate line based on a preset straight line direction and the endpoint coordinates;

[0231] determine two first target candidate lines with the same start point and different straight line directions;

[0232] determine two second target candidate lines with the same end point and different start points, respectively, of the two first target candidate lines;

[0233] determine the bounding line of the object to be measured based on the first target candidate line and the second target candidate line.

[0234] In a possible implementation, the second determining module 1103 is further configured to:

[0235] when there are a plurality of bounding boxes determined based on the first target candidate line and the second target candidate line, determine the bounding line of the object to be measured based on areas of the bounding boxes.

[0236] In a possible implementation, the second determining module 1103, after determining the bounding box of the object to be measured in the target image based on the endpoint coordinates, is further configured to:

[0237] In a case where the pattern formed by the bounding box of the object to be measured is not a rectangle, the bounding box of the object to be measured is corrected.

[0238] The description of the processing procedure of each module in the device and the interaction procedure between the modules can refer to the related description in the method embodiments, and will not be repeated here.

[0239] Based on the same technical concept, the embodiments of the present disclosure further provide a computer device. Referring to Figure 12 The structure diagram of the computer device 1200 provided by the embodiments of the present disclosure is shown in FIG. 12. The computer device 1200 includes a processor 1201, a memory 1202, and a bus 1203. The memory 1202 is configured to store execution instructions, including an internal memory 12021 and an external memory 12022. The internal memory 12021 is also called an internal memory, which is used to temporarily store operation data in the processor 1201 and exchange data with the external memory 12022 such as a hard disk. The processor 1201 exchanges data with the external memory 12022 through the internal memory 12021. When the computer device 1200 is running, the processor 1201 and the memory 1202 communicate through the bus 1203, so that the processor 1201 executes the following instructions:

[0240] obtaining a target image containing an object to be measured;

[0241] determining a plurality of first candidate lines in the target image based on an auxiliary region of the target image; wherein the auxiliary region is a preset region for displaying the object to be measured;

[0242] determining endpoint coordinates of the first candidate lines, and determining a bounding box of the object to be measured in the target image based on the endpoint coordinates;

[0243] using size information of the bounding box as size information of the object to be measured.

[0244] In a possible implementation, the instructions executed by the processor 1201 include that the determining a plurality of first candidate lines in the target image based on the auxiliary region of the target image includes:

[0245] determining a plurality of second candidate lines in the target image based on a pre-trained neural network;

[0246] determining a plurality of first candidate lines from the plurality of second candidate lines based on the auxiliary region of the target image.

[0247] In a possible implementation, the pre-trained neural network comprises a sampling network and a straight line verification network.

[0248] The pre-trained neural network is configured to determine the plurality of second candidate lines in the target image by:

[0249] The sampling network is configured to perform feature extraction on the target image to determine a feature map corresponding to the target image, and predict an intersection in the target image based on the sampling network.

[0250] The intersection is configured to determine a plurality of third candidate lines.

[0251] The plurality of third candidate lines, the feature map, and the straight line verification network are configured to determine the plurality of second candidate lines in the target image.

[0252] In a possible implementation, the pre-trained neural network comprises a sampling network and a straight line verification network.

[0253] For each of the plurality of third candidate lines, the sampling network is configured to determine a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line in the feature map.

[0254] The plurality of third candidate lines, the feature map, and the straight line verification network are configured to determine the plurality of second candidate lines in the target image.

[0255] In a possible implementation, the pre-trained neural network comprises a sampling network and a straight line verification network.

[0256] The sampling network is configured to determine a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line in the feature map.

[0257] The sampling network is configured to determine a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line in the feature map.

[0258] In a possible implementation, the pre-trained neural network comprises a sampling network and a straight line verification network.

[0259] obtaining a sample image carrying first annotation information and second annotation information, wherein the first annotation information is used to represent intersection points of object bounding lines in the sample image, and the second annotation information is used to represent the object bounding lines in the sample image;

[0260] inputting the sample image into the sampling network to obtain a feature map corresponding to the sample image and a predicted intersection point corresponding to the sample image;

[0261] determining a plurality of predicted candidate lines based on the predicted intersection point, and determining feature vectors corresponding to the plurality of predicted candidate lines based on the feature map;

[0262] determining a plurality of predicted lines based on the straight line verification network and the feature vectors corresponding to the plurality of predicted candidate lines;

[0263] determining a first loss value based on the predicted intersection point and the first annotation information, and determining a second loss value based on the predicted lines and the second annotation information;

[0264] training the neural network based on the first loss value and the second loss value.

[0265] In a possible implementation, in the instructions executed by the processor 1201, the determining, based on the auxiliary region of the target image, of a plurality of first candidate lines from the plurality of second candidate lines includes:

[0266] determining a length range for screening the first candidate lines based on a side length of the auxiliary region;

[0267] determining the first candidate lines from the plurality of second candidate lines based on the length range and a preset angle range.

[0268] In a possible implementation, in the instructions executed by the processor 1201, the determining, based on the endpoint coordinates, of the bounding line of the object to be measured in the target image includes:

[0269] for each first candidate line from the plurality of first candidate lines, determining a start point coordinate and an end point coordinate of the first candidate line based on a preset straight line direction and the endpoint coordinates;

[0270] determining two first target candidate lines with the same start point and different straight line directions;

[0271] determining two second target candidate lines with the same end point and different straight line directions, respectively starting from the end points of the two first target candidate lines;

[0272] determining the bounding line of the object to be measured based on the first target candidate line and the second target candidate line.

[0273] In a possible implementation, the method further includes, in the instructions executed by the processor 1201:

[0274] In a case where there are multiple detection boxes determined based on the first target candidate line and the second target candidate line, the frame line of the object to be measured is determined based on the area of each detection box.

[0275] In a possible implementation, the method further includes, in the instructions executed by the processor 1201:

[0276] In a case where the pattern formed by the frame line of the object to be measured is not a rectangle, the frame line of the object to be measured is corrected.

[0277] The embodiments of the present disclosure further provide a computer-readable storage medium, which stores a computer program. The computer program is run by a processor to perform the steps of the object size measurement method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0278] The embodiments of the present disclosure further provide a computer program product, which carries a program code. The program code includes instructions for performing the steps of the object size measurement method described in the above method embodiments. For details, refer to the above method embodiments, which will not be described here.

[0279] The computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) or the like.

[0280] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here. In several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and another division can be made in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0281] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0282] In addition, the functional units in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0283] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part of the prior art or the part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the various embodiments of the present disclosure. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0284] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit the present disclosure. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easy changes to the technical solutions described in the foregoing embodiments, or easily think of changes or equivalent replacements for some of the technical features; and these modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method of measuring the size of an object, characterized by, The method comprises: obtaining a target image containing an object to be measured; determining a first candidate line in the target image based on an auxiliary region of the target image, wherein the auxiliary region is a preset region for displaying the object to be measured; determining an endpoint coordinate of the first candidate line, and determining a frame line of the object to be measured in the target image based on the endpoint coordinate; taking size information of the frame line as size information of the object to be measured; the step of determining the first candidate line in the target image based on the auxiliary region of the target image comprises: determining a plurality of second candidate lines in the target image based on a pre-trained neural network; determining a first candidate line in the plurality of second candidate lines based on the auxiliary region of the target image; the pre-trained neural network comprises a sampling network and a straight line verification network; the structure of the straight line verification network is that a first activation function is stacked on a first linear layer at the bottom, a second linear layer is stacked on the first activation function, a second activation function is stacked on the second linear layer, a third linear layer is stacked on the second activation function, and a third activation function is stacked on the third linear layer; the step of determining the plurality of second candidate lines in the target image based on the pre-trained neural network comprises: performing feature extraction on the target image based on the sampling network to determine a feature map corresponding to the target image, and predicting an intersection point in the target image based on the sampling network; determining a plurality of third candidate lines based on the intersection point; for each third candidate line in the plurality of third candidate lines, determining a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line on the feature map; inputting the feature vectors corresponding to the plurality of third candidate lines obtained respectively into the straight line verification network one by one to obtain the plurality of second candidate lines in the target image; the step of inputting the feature vectors corresponding to the plurality of third candidate lines obtained respectively into the straight line verification network one by one to obtain the plurality of second candidate lines in the target image comprises: inputting the feature vector corresponding to the third candidate line into the straight line verification network to obtain a confidence degree corresponding to the third candidate line respectively; wherein the confidence degree represents a probability that the third candidate line is an object edge; taking the third candidate line satisfying a preset condition as the second candidate line.

2. The method of claim 1, wherein, The method further comprises the following steps for training the neural network: obtaining a sample image carrying first annotation information and second annotation information, wherein the first annotation information is used to represent an intersection point of an object frame line in the sample image; and the second annotation information is used to represent the object frame line in the sample image; inputting the sample image into the sampling network to obtain a feature map corresponding to the sample image and a predicted intersection point corresponding to the sample image; determining a plurality of predicted candidate lines based on the predicted intersection point, and determining a feature vector corresponding to each of the plurality of predicted candidate lines based on the feature map; determine a plurality of predicted lines based on the straight line verification network and the feature vectors corresponding to the plurality of predicted candidate lines respectively; determine a first loss value based on the predicted intersection point and the first annotation information, and determine a second loss value based on the predicted lines and the second annotation information; train the neural network based on the first loss value and the second loss value.

3. The method of any of claim 1, wherein, The determining, based on the auxiliary region of the target image, of the first candidate line from the plurality of second candidate lines includes: determining a length range for screening the first candidate line based on the side length of the auxiliary region; determining the first candidate line from the plurality of second candidate lines based on the length range and a preset angle range.

4. The method of claim 1, wherein, The determining, based on the endpoint coordinates, of the bounding line of the object to be measured in the target image includes: for each first candidate line from the plurality of first candidate lines, determining the start point coordinates and the end point coordinates of the first candidate line based on a preset straight line direction and the endpoint coordinates; determining two first target candidate lines that have the same start point and different straight line directions; determining two second target candidate lines that have the same end point and are respectively started from the end points of the two first target candidate lines; determining the bounding line of the object to be measured based on the first target candidate line and the second target candidate line.

5. The method of claim 4, wherein, The method further includes: in a case where there are multiple detection boxes determined based on the first target candidate line and the second target candidate line, determining the bounding line of the object to be measured based on the areas of the detection boxes.

6. The method of claim 1, wherein, After determining the bounding line of the object to be measured in the target image based on the endpoint coordinates, the method further includes: in a case where the pattern formed by the bounding line of the object to be measured is not a rectangle, correcting the bounding line of the object to be measured.

7. An object size measurement device, characterized by, includes: an acquisition module configured to acquire a target image containing an object to be measured; a first determination module configured to determine a first candidate line in the target image based on an auxiliary region of the target image; the auxiliary region is a preset region for displaying the object to be measured; a second determination module configured to determine endpoint coordinates of the first candidate line and determine a bounding line of the object to be measured in the target image based on the endpoint coordinates; a size measurement module configured to take size information of the bounding line as size information of the object to be measured; The determining, based on the auxiliary region of the target image, of the first candidate line in the target image includes: determining a plurality of second candidate lines in the target image based on a pre-trained neural network; determining a first candidate line from the plurality of second candidate lines based on the auxiliary region of the target image; The pre-trained neural network includes a sampling network and a straight line verification network; the structure of the straight line verification network is that a first activation function is stacked on a first linear layer at the bottom, a second linear layer is stacked on the first activation function, a second activation function is stacked on the second linear layer, a third linear layer is stacked on the second activation function, and a third activation function is stacked on the third linear layer; The determining, based on the pre-trained neural network, of the plurality of second candidate lines in the target image includes: perform feature extraction on the target image based on the sampling network, determine a feature map corresponding to the target image, and predict an intersection point in the target image based on the sampling network; determine a plurality of third candidate lines based on the intersection point; for each third candidate line in the plurality of third candidate lines, determine a feature vector corresponding to the third candidate line based on feature values corresponding to points on the third candidate line in the feature map; input the feature vectors corresponding to the plurality of third candidate lines into the straight line verification network one by one to obtain a plurality of second candidate lines in the target image; the inputting the feature vectors corresponding to the plurality of third candidate lines into the straight line verification network one by one to obtain the plurality of second candidate lines in the target image comprises: input the feature vector corresponding to the third candidate line into the straight line verification network to obtain a confidence score corresponding to the third candidate line, wherein the confidence score represents a probability that the third candidate line is an object edge; take the third candidate line whose confidence score meets a preset condition as the second candidate line.

8. A computer device, comprising: comprise: a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the object size measurement method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the object size measurement method in any one of claims 1 to 6.

Images