Shelf line detection method and device, computer device and storage medium

Abstract

The application relates to a shelf line detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a target image; determining a first position label and a second position label corresponding to each pixel point in the target image through a trained target detection model; the first position label is used for representing whether the pixel point is an internal pixel point of a shelf line, and the second position label is used for representing whether the pixel point is a head pixel point or a tail pixel point; determining at least one shelf line pixel point set based on the first position label of each pixel point in the target image; in each shelf line pixel point set, a head pixel point set and a tail pixel point set are respectively determined based on the second position label of each pixel point; four shelf line vertices of a corresponding shelf line in the target image are determined based on each head pixel point set and a corresponding tail pixel point set, and the shelf line is positioned from the target image based on the shelf line vertices. The method can improve the detection accuracy of the shelf line.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a shelf line detection method and device, a computer device and a storage medium. BACKGROUND

[0002] With the continuous development of computer vision technology, computer vision technology is widely used in various industries. For example, in the new retail industry, computer vision technology is used to analyze shelf pictures and identify product information on the shelves to achieve automatic product display and inventory checking. As a carrier of products, accurately detecting the shelf line from the shelf picture is of great significance for correcting the shelf picture and refining the product location information.

[0003] Currently, the straight line segments in the shelf picture are usually detected by traditional image processing methods such as binarization and edge detection, and the position information of the shelf line is estimated based on the detected straight line segments. However, based on this detection method, in addition to the shelf line in the shelf picture, straight line segments in the products, ceiling and floor will also be detected, which will affect the determination of the position information of the shelf line, and there are problems of low detection accuracy and poor stability. SUMMARY

[0004] Therefore, it is necessary to provide a shelf line detection method, device, computer device and storage medium capable of improving the detection accuracy of the shelf line to solve the above technical problems.

[0005] A shelf line detection method, the method comprising:

[0006] obtaining a target image;

[0007] determining a first position label and a second position label corresponding to each pixel point in the target image by a trained target detection model; the first position label is used to represent whether the corresponding pixel point is a shelf line internal pixel point, and the second position label is used to represent whether the corresponding pixel point is a head pixel point or a tail pixel point;

[0008] determining at least one shelf line pixel point set based on the first position label corresponding to each pixel point in the target image;

[0009] determining a head pixel point set and a corresponding tail pixel point set based on the second position label corresponding to each pixel point in each shelf line pixel point set;

[0010] determining four shelf line vertices corresponding to the corresponding shelf line in the target image based on each head pixel point set and the corresponding tail pixel point set, and positioning the shelf line from the target image based on the shelf line vertices.

[0011] In one of the embodiments, the determining the at least one shelf line pixel point set based on the first position label corresponding to each pixel point in the target image comprises:

[0012] By traversing the pixel points in the target image, the pixel points in each row with the first position label as the shelf line internal pixel point and the adjacent pixel points are clustered to obtain a shelf line pixel point subset, and the shelf line pixel point subsets located in adjacent two rows and having at least one pixel point with the same horizontal coordinate are merged to obtain the at least one shelf line pixel point set.

[0013] In one of the embodiments, the pixel point corresponding to the second position label as the head pixel point has an offset, and the pixel point corresponding to the second position label as the tail pixel point has an offset. The determining the four shelf line vertices corresponding to the corresponding shelf line in the target image based on each head pixel point set and the corresponding tail pixel point set, and the positioning the shelf line from the target image based on the shelf line vertices comprises:

[0014] Determining two shelf line vertices in the corresponding shelf line head region based on the offset corresponding to each pixel point in each head pixel point set.

[0015] Determining two shelf line vertices in the corresponding shelf line tail region based on the offset corresponding to each pixel point in each tail pixel point set.

[0016] Positioning the corresponding shelf line from the target image based on the shelf line vertex in each shelf line head region and the shelf line vertex in the corresponding shelf line tail region.

[0017] In one of the embodiments, the training step of the target detection model comprises:

[0018] Obtaining a training sample set; the training sample set comprises a sample image and a sample feature set corresponding to the sample image; the sample feature set comprises a first sample position label and a second sample position label corresponding to each pixel point in the sample image, and a sample offset corresponding to the pixel point with the second sample position label as the head pixel point or the tail pixel point;

[0019] Performing model training based on the training sample set to obtain the trained target detection model.

[0020] In one of the embodiments, the training step of the target detection model further comprises:

[0021] Detecting the sample image by the target detection model to be trained to obtain a predicted feature set corresponding to the sample image;

[0022] determine a target loss function according to the sample feature set and the prediction feature set, and adjust model parameters of the target detection model to be trained reversely based on the target loss function;

[0023] perform the detection of the sample image by the target detection model to be trained to obtain the prediction feature set corresponding to the sample image until iteration is stopped, and obtain the trained target detection model.

[0024] In one of the embodiments, the prediction feature set comprises a first prediction position label and a second prediction position label corresponding to each pixel point in the sample image, and a prediction offset corresponding to each pixel point whose second prediction position label is a head pixel point or a tail pixel point; and the determining of the target loss function according to the sample feature set and the prediction feature set comprises:

[0025] determining a first loss function based on the first prediction position label corresponding to each pixel point in the sample image and the first sample position label, determining a second loss function based on the second prediction position label corresponding to each pixel point in the sample image and the second sample position label, and determining a third loss function based on the prediction offset corresponding to each pixel point whose second prediction position label is a head pixel point or a tail pixel point in the sample image and the sample offset corresponding to each pixel point whose second sample position label is a head pixel point or a tail pixel point;

[0026] obtaining a corresponding target loss function according to a preset weight based on the first loss function, the second loss function and the third loss function.

[0027] In one of the embodiments, the obtaining of the training sample set comprises:

[0028] obtaining a sample image and a sample shelf line vertex corresponding to each sample shelf line in each sample image;

[0029] preprocessing the sample shelf line vertex corresponding to each sample shelf line to obtain a corresponding sample head region and a sample tail region;

[0030] determining a corresponding sample feature set based on the sample head region and the sample tail region corresponding to each sample shelf line in each sample image;

[0031] obtaining a training sample set according to the sample image and the sample feature set corresponding to each sample image.

[0032] A shelf line detection device, the device comprising:

[0033] an obtaining module configured to obtain a target image;

[0034] The label determination module is configured to determine a first position label and a second position label corresponding to each pixel point in the target image; the first position label is used to represent whether the corresponding pixel point is an internal pixel point of a shelf line, and the second position label is used to represent whether the corresponding pixel point is a head pixel point or a tail pixel point.

[0035] The first set determination module is configured to determine at least one shelf line pixel point set based on the first position label corresponding to each pixel point in the target image.

[0036] The second set determination module is configured to determine, in each shelf line pixel point set, a head pixel point set and a corresponding tail pixel point set based on the second position label corresponding to each pixel point.

[0037] The detection module is configured to determine four shelf line vertices corresponding to a corresponding shelf line in the target image based on each head pixel point set and the corresponding tail pixel point set, and to locate the shelf line from the target image based on the shelf line vertices.

[0038] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps in each of the above method embodiments when executing the computer program.

[0039] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in each of the above method embodiments.

[0040] The above shelf line detection method, device, computer device, and storage medium can improve the accuracy of each position label by determining the first position label and the second position label corresponding to each pixel point in the target image through a trained target detection model. The shelf line pixel point set including the pixel points belonging to the same shelf line is determined based on the first position label corresponding to each pixel point, i.e., the shelf line pixel point set corresponding to each shelf line in the target image is determined based on the first position label. The head pixel point set and the tail pixel point set of the corresponding shelf line are determined based on the second position label corresponding to each pixel point in each shelf line pixel point set, which can improve the accuracy of the head and tail pixel point sets. Furthermore, the four shelf line vertices corresponding to the shelf line in the target image are determined based on the pixel points in the head pixel point set and the tail pixel point set corresponding to each shelf line and having high accuracy, and the corresponding shelf line is located from the target image based on the four shelf line vertices, which can improve the detection accuracy of the shelf line. The shelf line vertices are determined based on the pixel points in the head and tail pixel point sets corresponding to each shelf line, so as to locate the corresponding shelf line based on the shelf line vertices, which can further improve the detection accuracy of the shelf line. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating a shelf line detection method in one embodiment;

[0042] Figure 2 This is a schematic diagram illustrating the determination of the head and tail regions based on the vertices of the shelving line in one embodiment.

[0043] Figure 3 This is a flowchart illustrating the shelf line detection method in another embodiment;

[0044] Figure 4 This is a schematic diagram illustrating the principle of target image detection based on a target detection model in one embodiment;

[0045] Figure 5 This is a schematic diagram illustrating the shelf line detection effect implemented based on the shelf line detection method in one embodiment;

[0046] Figure 6 This is a structural block diagram of a shelf line detection device in one embodiment;

[0047] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] In one embodiment, such as Figure 1 As shown, a shelf line detection method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0050] Step 102: Obtain the target image.

[0051] The target image is an image of the shelf line to be inspected. Specifically, the terminal acquires the target image through an image acquisition device, such as a camera or webcam. The image acquisition device can be built into the terminal as a component or connected externally as a standalone device. The image acquisition device can communicate with the terminal via wired or wireless means.

[0052] In step 104, the first position label and the second position label corresponding to each pixel point in the target image are determined by the trained target detection model. The first position label is used to represent whether the corresponding pixel point is an internal pixel point of the shelf line, and the second position label is used to represent whether the corresponding pixel point is a head pixel point or a tail pixel point.

[0053] The target detection model is a model that can be used to determine the first position label and the second position label corresponding to each pixel point in the target image, which is trained based on a pre-acquired training sample set. The training sample set includes a sample image and a sample feature set corresponding to each sample image. The sample feature set includes a first sample position label and a second sample position label corresponding to each pixel point in the sample image, and a sample offset corresponding to a pixel point whose second sample position label is a head pixel point or a tail pixel point. The first position label corresponding to a pixel point is used to represent whether the pixel point is an internal pixel point of the shelf line. The first position label can specifically include internal pixel points of the shelf line and non-internal pixel points of the shelf line. The first position label can also be a numerical value used to determine whether the corresponding pixel point is an internal pixel point of the shelf line, so that based on the first position label and a preset numerical threshold, it can be determined whether the corresponding pixel point is an internal pixel point of the shelf line. If the first position label is greater than or equal to the preset numerical threshold, it is determined that the corresponding pixel point is an internal pixel point of the shelf line, otherwise, it is determined that the corresponding pixel point is a non-internal pixel point of the shelf line. The internal pixel point of the shelf line refers to a pixel point in the shelf line region, which can specifically be a pixel point in the internal shelf line or a pixel point at the edge of the shelf line.

[0054] The second position label corresponding to a pixel point is used to represent whether the pixel point is a head pixel point or a tail pixel point of the shelf line. The second position label can specifically be a head pixel point, a tail pixel point, and a non-head-tail pixel point. The second position label can also be a numerical value used to determine whether the corresponding pixel point is a head pixel point or a tail pixel point, so that based on the second position label, a first numerical threshold and a second numerical threshold, it can be determined whether the corresponding pixel point is a head-tail pixel point. If the second position label is greater than or equal to the first numerical threshold, it is determined that the corresponding pixel point is a tail pixel point. If the second position label is less than or equal to the second numerical threshold, it is determined that the corresponding pixel point is a head pixel point. Otherwise, it is determined that the corresponding pixel point is a non-head-tail pixel point. The sum of the first numerical threshold and the second numerical threshold can be 1. For example, the first numerical threshold is 0.9, and the second numerical threshold is 0.1. The head pixel point refers to a pixel point in the head region of the shelf line. The tail pixel point refers to a pixel point in the tail region of the shelf line. The head pixel point and the tail pixel point can be collectively referred to as head-tail pixel points.

[0055] Specifically, after obtaining the target image of the to-be-detected shelf line, the terminal inputs the target image into the trained target detection model, detects the target image through the target detection model, and obtains the first position label and the second position label corresponding to each pixel point in the target image, so as to detect the shelf line based on the first position label and the second position label corresponding to each pixel point.

[0056] In step 106, at least one shelf line pixel point set is determined based on the first position label corresponding to each pixel point in the target image.

[0057] The shelf line pixel point set is a pixel point set composed of pixel points on a single shelf line, that is, a pixel point set composed of pixel points in a same shelf line region, and each shelf line in the target image corresponds to a shelf line region.

[0058] Specifically, after obtaining the first position label and the second position label corresponding to each pixel point in the target image, the terminal determines the shelf line pixel point set corresponding to each shelf line in the target image according to the first position label corresponding to each pixel point through a non-maximum suppression method, that is, determines at least one shelf line pixel point set corresponding to the target image, and each shelf line pixel point set corresponds to a shelf line. It can be understood that if the target image does not include a shelf line, the terminal cannot determine the shelf line pixel point set corresponding to the target image, and will not perform subsequent shelf line detection operations on the target image.

[0059] In one embodiment, the terminal traverses each pixel point in the target image, and determines the shelf line pixel point set corresponding to each shelf line in the target image based on the first position label corresponding to the traversed pixel point. Specifically, the terminal traverses the pixel points in the same row in the target image in sequence, clusters the pixel points in the same shelf line region in each row according to the first position label corresponding to each pixel point, obtains the corresponding shelf line pixel point subset, and merges the multiple shelf line pixel point subsets in the same shelf line region in each row by traversing the shelf line pixel point subsets in each row, to obtain the corresponding shelf line pixel point set. Wherein, the row in the target image refers to a single pixel row in the target image. The upper left corner of the target image is taken as the coordinate origin, the horizontal right direction is taken as the horizontal axis, and the vertical downward direction is taken as the vertical axis. Thus, the row in the target image refers to a pixel row composed of pixel points with the same vertical coordinate in the target image.

[0060] Further, each of the subset of shelf line pixels includes at least one first position label of the pixel points inside the shelf line, when the subset of shelf line pixels includes multiple pixel points, each of the multiple pixel points has at least one adjacent pixel point in the subset of shelf line pixels, that is, the multiple pixel points in the single subset of shelf line pixels are continuous pixel points in the same row. If multiple subsets of shelf line pixels exist in a single row, the pixel points between any two of the multiple subsets of shelf line pixels are not adjacent to each other, that is, each pixel point in each subset of shelf line pixels is not adjacent to each pixel point in each other subset of shelf line pixels in the same row. The multiple subsets of shelf line pixels in each set of shelf line pixels are in different rows in the target image, and each subset of shelf line pixels has at least one adjacent subset of shelf line pixels in the multiple subsets of shelf line pixels. Two pixel points in the same row are adjacent, which means that the two pixel points are continuous pixel points in the same row. Two subsets of shelf line pixels are adjacent, which means that the two subsets of shelf line pixels are in adjacent rows in the target image, and at least one pixel point with the same horizontal coordinate exists in the two subsets of shelf line pixels.

[0061] It can be understood that, since the shelf lines in the target image can be displayed obliquely, each shelf line can span multiple rows of the target image, and a same row of the target image can include multiple shelf line regions belonging to different shelf lines. Since the multiple shelf lines in the target image do not overlap with each other, the multiple shelf line regions belonging to different shelf lines in the same row are not adjacent to each other. Therefore, if multiple subsets of shelf line pixels exist in a single row of the target image, it indicates that multiple shelf line regions belonging to different shelf lines exist in the single row, each shelf line region corresponds to a subset of shelf line pixels, and the multiple subsets of shelf line pixels are not adjacent to each other. For a single shelf line spanning multiple rows of the target image, since the shelf line is continuous and spans multiple rows of the target image, the multiple subsets of shelf line pixels corresponding to the shelf line are continuous and are in multiple rows of the target image, and at least one pair of upper and lower adjacent pixel points exists in two subsets of shelf line pixels in adjacent rows.

[0062] In step 108, in each set of shelf line pixels, a set of head pixel points and a corresponding set of tail pixel points are determined based on the second position labels of the pixel points.

[0063] Specifically, after obtaining the at least one set of shelf line pixel points corresponding to the target image, the terminal clusters each set of shelf line pixel points according to the second position labels corresponding to the pixel points in the set of shelf line pixel points, to obtain a corresponding set of head pixel points and a set of tail pixel points, that is, the pixel points with the second position label of head pixel points in the set of shelf line pixel points are clustered into the set of head pixel points, and the pixel points with the second position label of tail pixel points in the set of shelf line pixel points are clustered into the set of tail pixel points. Each set of shelf line pixel points corresponds to a shelf line in the target image. Thus, the set of head region pixel points corresponding to the head region of each shelf line and the set of tail region pixel points corresponding to the tail region of each shelf line can be obtained in the above manner.

[0064] In step 110, four shelf line vertices corresponding to the corresponding shelf line in the target image are determined based on each set of head pixel points and the corresponding set of tail pixel points, and the shelf line is located from the target image based on the shelf line vertices.

[0065] Specifically, after determining the set of head pixel points and the set of tail pixel points corresponding to each shelf line in the target image, for each shelf line, two shelf line vertices in the head region of the shelf line are determined according to the pixel points in the set of head pixel points corresponding to the shelf line, and two shelf line vertices in the tail region of the shelf line are determined according to the pixel points in the set of tail pixel points corresponding to the shelf line, that is, four shelf line vertices of the shelf line in the target image are determined. After obtaining the four shelf line vertices of each shelf line, the corresponding shelf line can be located from the target image based on the four shelf line vertices of each shelf line.

[0066] It can be understood that the shelf line in the target image is usually long strip-shaped, that is, each shelf line usually has a certain width and length, so that each shelf line corresponds to four shelf line vertices in the target image, and the corresponding shelf line can be located in the target image based on the four shelf line vertices. The four shelf line vertices of each shelf line can be understood as the key points of the shelf line, and by locating the key points of the shelf line, the shelf line can be located from the target image.

[0067] The above shelf line detection method can improve the accuracy of each position label by determining the first position label and the second position label corresponding to each pixel point in the target image through the trained target detection model, and obtain position labels with high accuracy. Based on the first position label with high accuracy corresponding to each pixel point, the shelf line pixel point set including the pixel points belonging to the same shelf line is determined, that is, the shelf line pixel point set corresponding to each shelf line in the target image is determined based on the first position label, and based on the second position label with high accuracy corresponding to each pixel point in each shelf line pixel point set, the head pixel point set and the tail pixel point set of the corresponding shelf line are determined, which can improve the accuracy of the head and tail pixel point sets. Further, based on each pixel point in the head pixel point set and the tail pixel point set corresponding to each shelf line and with high accuracy, the four shelf line vertices corresponding to the shelf line in the target image are determined, and then the corresponding shelf line is located from the target image based on the four shelf line vertices, which can improve the detection accuracy of the shelf line. Based on each pixel point in the head and tail pixel point sets corresponding to each shelf line, the shelf line vertices are determined correspondingly to facilitate the positioning of the corresponding shelf line by the shelf line vertices, which can further improve the detection accuracy of the shelf line.

[0068] In one embodiment, step 106 includes: by traversing the pixel points in the target image, clustering the adjacent pixel points with the first position label as the shelf line internal pixel points in each row to obtain a shelf line pixel point subset, and merging the shelf line pixel point subsets located in the adjacent two rows and having at least one pixel point with the same horizontal coordinate to obtain at least one shelf line pixel point set.

[0069] In the embodiment, the upper left corner of the target image is taken as the coordinate origin, the direction to the right horizontally is taken as the horizontal axis, and the direction downward vertically is taken as the vertical axis. Thus, the row in the target image refers to the pixel row composed of the pixel points with the same vertical coordinate in the target image.

[0070] Specifically, the terminal traverses each pixel point in the target image in turn according to rows, and when traversing each pixel point in a same row in turn, clusters, according to the first position label corresponding to each pixel point, the pixel point in the current traversed row whose first position label is a shelf line internal pixel point, according to the pixel point position of each pixel point in the target image, to cluster the pixel points adjacent in pixel point position to the same shelf line pixel point subset, to obtain one or more shelf line pixel point subsets corresponding to the current traversed row. It can be understood that if there is no pixel point in the current traversed row whose first position label is a shelf line internal pixel point, then the current traversed row does not have a corresponding shelf line pixel point subset.

[0071] Further, for the plurality of shelf line pixel point subsets distributed in adjacent two rows of the target image, the terminal determines whether there is at least one pixel point with the same horizontal coordinate between any two shelf line pixel point subsets, merges the two shelf line pixel point subsets determined to have at least one pixel point with the same horizontal coordinate, to obtain a merged shelf line pixel point subset, and continues to merge with the shelf line pixel point subsets of other adjacent rows according to the above merging manner for the merged shelf line pixel point subset, until there is no shelf line pixel point subset that can be continuously merged in the target image, to obtain a corresponding shelf line pixel point set. Thus, the terminal merges each shelf line pixel point subset in the target image according to the above merging manner, to obtain a corresponding at least one shelf line pixel point set. It can be understood that in the merging process of the shelf line pixel point subset, the terminal determines, for each shelf line pixel point subset, whether the shelf line pixel point subset needs to be merged with each other shelf line pixel point subset in the adjacent row, and merges according to the determination result.

[0072] In one embodiment, the terminal can merge each shelf line pixel point subset according to the above merging manner after traversing each pixel point in the target image and obtaining the shelf line pixel point subset corresponding to each row, to obtain a corresponding shelf line pixel point set. The terminal can also merge the currently obtained shelf line pixel point subset with the shelf line pixel point subset obtained based on the row of the previous traversal according to the above merging manner after traversing each pixel point in the current traversed row and obtaining the shelf line pixel point subset corresponding to the current traversed row, and continue to traverse the next row, perform the above merging operation on the shelf line pixel point subset corresponding to the row of the next traversal, until each row in the target image is traversed, and the above merging operation is performed on the shelf line pixel point subset corresponding to each row, to obtain at least one shelf line pixel point set corresponding to the target image.

[0073] In the above embodiment, by traversing the pixel points in the target image, the pixel points in each row within the same shelf line region are clustered to obtain a shelf line pixel point subset, and the shelf line pixel point subsets within the same shelf line region are merged to obtain a corresponding shelf line pixel point set, so as to facilitate the shelf line detection based on the pixel points in each shelf line pixel point set, and the shelf line detection accuracy can be improved.

[0074] In one embodiment, the pixel point corresponds to an offset amount when the second position label of the pixel point in the target image is a head pixel point; the pixel point corresponds to an offset amount when the second position label of the pixel point in the target image is a tail pixel point; step 110 comprises: determining two shelf line vertices in the corresponding shelf line head region based on the offset amount corresponding to each pixel point in each head pixel point set; determining two shelf line vertices in the corresponding shelf line tail region based on the offset amount corresponding to each pixel point in each tail pixel point set; and positioning the corresponding shelf line from the target image based on the shelf line vertices in each shelf line head region and the shelf line vertices in the corresponding shelf line tail region.

[0075] In one embodiment, the pixel point corresponds to an offset amount when the second position label of the pixel point in the target image is a head pixel point; the pixel point corresponds to an offset amount when the second position label of the pixel point in the target image is a tail pixel point; step 110 comprises: determining two shelf line vertices in the corresponding shelf line head region based on the offset amount corresponding to each pixel point in each head pixel point set; determining two shelf line vertices in the corresponding shelf line tail region based on the offset amount corresponding to each pixel point in each tail pixel point set; and positioning the corresponding shelf line from the target image based on the shelf line vertices in each shelf line head region and the shelf line vertices in the corresponding shelf line tail region.

[0076] Specifically, the terminal determines, by the trained target detection model, the first position label and the second position label corresponding to each pixel point in the target image, and determines, by the trained target detection model, the offset corresponding to each pixel point with the second position label being a head pixel point or a tail pixel point in the target image. After obtaining the head pixel point set and the tail pixel point set corresponding to each shelf line in the target image in the above manner, the terminal determines, for each shelf line, two shelf line vertices in the head region of the shelf line according to the offset corresponding to each pixel point in the head pixel point set corresponding to the shelf line, and determines two shelf line vertices in the tail region of the shelf line according to the offset corresponding to each pixel point in the tail pixel point set corresponding to the shelf line. Further, the terminal locates the shelf line from the target image according to the two shelf line vertices in the head region and the two shelf line vertices in the tail region of each shelf line.

[0077] In one embodiment, the terminal inputs the target image into the trained target detection model, detects the target image by the target detection model, and obtains the first position label and the second position label corresponding to each pixel point in the target image, and the offset corresponding to each pixel point with the second position label being a head pixel point or a tail pixel point.

[0078] In one embodiment, for each shelf line, the terminal can obtain the plurality of predicted position coordinates corresponding to each of the two shelf line vertices according to the position coordinates of each pixel point in the head pixel point set corresponding to the shelf line in the target image and the four offsets of each pixel point relative to the two shelf line vertices in the head region of the shelf line, and obtain the position coordinates of each shelf line vertex according to the plurality of predicted position labels corresponding to each shelf line vertex.

[0079] Specifically, for each pixel point in the head pixel point set corresponding to a single shelf line, the terminal can calculate a predicted position coordinate corresponding to each of the two shelf line vertices based on the position coordinate of the pixel point and the four offsets of the pixel point relative to the corresponding two shelf line vertices. Thus, based on multiple pixel points in the head pixel point set, multiple predicted position coordinates corresponding to each of the two shelf line vertices can be obtained. The position coordinate includes the horizontal coordinate and the vertical coordinate. Further, for each shelf line vertex, the terminal averages the horizontal coordinates in the multiple predicted position coordinates corresponding to the shelf line vertex to obtain the horizontal coordinate of the shelf line vertex, and averages the vertical coordinates in the multiple predicted position coordinates corresponding to the shelf line vertex to obtain the vertical coordinate of the shelf line vertex. Thus, based on the obtained horizontal coordinate and vertical coordinate, the position coordinate of the shelf line vertex in the target image is obtained, that is, the shelf line vertex is determined from the target image. The average can be arithmetic average or weighted average. It can be understood that the terminal can determine the two shelf line vertices in the tail region of the shelf line based on the tail pixel point set corresponding to the shelf line in a similar manner as described above, and will not be described here.

[0080] For example, assuming that the position coordinate of a pixel point in the head pixel point set is (x, y), and the offsets of the pixel point are (p1, p2, p3, p4), that is, the offsets of the pixel point relative to the two shelf line vertices in the corresponding head region are (p1, p2) and (p3, p4), respectively. Thus, based on the position coordinate of the pixel point and the offsets of the pixel point relative to each shelf line vertex, the predicted position coordinates corresponding to each shelf line vertex can be determined as (p1+x, p2+y) and (p3+x, p4+y), respectively.

[0081] In the above embodiment, the shelf line vertices in the head region can be determined based on the offsets of the pixel points in the head pixel point set relative to the shelf line vertices in the head region, and the shelf line vertices in the tail region can be determined based on the offsets of the pixel points in the tail pixel point set relative to the shelf line vertices in the tail region, which can improve the positioning accuracy of the shelf line vertices, so that the corresponding shelf line can be accurately positioned based on the shelf line vertices with higher accuracy.

[0082] In one embodiment, the training step of the target detection model includes: obtaining a training sample set; the training sample set includes a sample image and a sample feature set corresponding to the sample image; the sample feature set includes a first sample position label and a second sample position label corresponding to each pixel point in the sample image, and a sample offset corresponding to a pixel point whose second sample position label is a head pixel point or a tail pixel point; based on the training sample set, the model is trained to obtain a trained target detection model.

[0083] The sample feature set is a set composed of feature data corresponding to each pixel point in a sample image. Each sample image corresponds to a sample feature set. The feature data corresponding to each pixel point includes a first sample position label and a second sample position label corresponding to the pixel point, the second sample position label being feature data corresponding to a head pixel point or a tail pixel point, and a sample offset corresponding to the pixel point.

[0084] Specifically, the terminal obtains a plurality of sample images, and determines a corresponding first sample position label and a second sample position label for each pixel point in each sample image, and determines a corresponding sample offset for a pixel point with a second sample position label being a head pixel point or a tail pixel point in the sample image. The terminal obtains a sample feature set corresponding to each sample image according to the first sample position label, the second sample position label, and the sample offset corresponding to each pixel point in the sample image, and obtains a corresponding training sample set according to the plurality of sample images and the sample feature set corresponding to each sample image. Further, the terminal performs model training by taking a sample image in the training sample set as an input feature and taking a corresponding sample feature set as an expected output feature, to obtain a trained target detection model.

[0085] In an embodiment, the machine learning algorithm involved in training the target detection model by the terminal includes, but is not limited to, a fully convolutional network.

[0086] In the above embodiment, the trained target detection model is obtained based on the labeled training sample set, so that when performing shelf line detection, the trained target detection model can quickly and accurately determine the feature data corresponding to each pixel point in a target image, and perform shelf line detection based on the feature data, thereby improving the detection accuracy and efficiency of the shelf line.

[0087] In an embodiment, the training step of the target detection model further includes: detecting the sample image by the target detection model to be trained to obtain a predicted feature set corresponding to the sample image; determining a target loss function according to the sample feature set and the predicted feature set, and adjusting the model parameters of the target detection model to be trained based on the target loss function; performing the step of detecting the sample image by the target detection model to be trained to obtain a predicted feature set corresponding to the sample image until stopping iteration, to obtain the trained target detection model.

[0088] Specifically, the terminal inputs the sample images as input features into the target detection model to be trained, detects the sample images through the target detection model to be trained, and obtains a corresponding prediction feature set. The terminal determines a target loss function according to the prediction feature set corresponding to each sample image and the sample feature set, and reversely adjusts the model parameters of the target detection model to be trained according to the target loss function, to obtain a target detection model with adjusted model parameters. The terminal inputs the sample images as input features into the target detection model with adjusted model parameters, obtains a corresponding prediction feature set, and continues to reversely adjust the model parameters of the target detection model with adjusted model parameters according to the prediction feature set and the corresponding sample feature set, and so on, until an iteration stop condition is met, and the iteration is stopped, to obtain a trained target detection model. The iteration stop condition is, for example, that the number of iterations is greater than or equal to a preset number of iterations, or that the sample images in the training sample set are all used for model training, which is not specifically limited here.

[0089] In one embodiment, in the training process of the target detection model, the model parameters of the target detection model to be trained are reversely adjusted based on the target loss function determined by the sample feature set and the corresponding prediction feature set, so that the function value of the target loss function is continuously reduced, and thus the detection accuracy of the target detection model to be trained is increasingly improved.

[0090] In one embodiment, the terminal groups the sample images in the training sample set to obtain a plurality of sample image groups, and in each iteration training process, inputs the plurality of sample images in each sample image group as input features into the target detection model to be trained at the same time, obtains a prediction feature set corresponding to each of the plurality of sample images, and then obtains a corresponding target loss function based on the prediction feature set corresponding to each of the plurality of sample images and the sample feature set.

[0091] In the above embodiments, in the model training process, the model parameters of the target detection model are reversely adjusted based on the target loss function determined by the prediction feature set and the sample feature set, so as to continuously improve the detection accuracy of the target detection model through iteration training.

[0092] In one embodiment, the prediction feature set includes the first prediction position label and the second prediction position label corresponding to each pixel point in the sample image, and the prediction offset corresponding to each pixel point with the second prediction position label being a head pixel point or a tail pixel point; the target loss function is determined according to the sample feature set and the prediction feature set, including: determining the first loss function based on the first prediction position label corresponding to each pixel point in the sample image and the first sample position label, determining the second loss function based on the second prediction position label corresponding to each pixel point in the sample image and the second sample position label, and determining the third loss function based on the prediction offset corresponding to each pixel point with the second prediction position label being a head pixel point or a tail pixel point in the sample image, and the sample offset corresponding to each pixel point with the second sample position label being a head pixel point or a tail pixel point; and obtaining the corresponding target loss function according to the preset weight based on the first loss function, the second loss function and the third loss function.

[0093] The preset weight is a weight preset in advance. The first loss function, the second loss function and the third loss function are respectively pre-configured with corresponding preset weights, and the sum of the three preset weights is not limited to 1, for example, the preset weight of the first loss function is 1, and the preset weights of the second loss function and the third loss function are 0.6 and 0.7 respectively, which are not limited in detail.

[0094] Specifically, in the training process of the target detection model, the terminal detects the sample image through the target detection model to be trained to obtain a prediction feature set corresponding to the sample image, each sample image corresponds to a prediction feature set, and the prediction feature set corresponding to the sample image includes the first prediction position label and the second prediction position label corresponding to each pixel point in the sample image, and the prediction offset corresponding to each pixel point with the second prediction position label being a head pixel point or a tail pixel point in the sample image. Further, the terminal determines the first loss function according to the first prediction position label corresponding to each pixel point in the sample image and the first sample position label according to the class balanced cross-entropy loss calculation method, and determines the second loss function according to the second prediction position label corresponding to each pixel point in the sample image and the second sample position label according to the class balanced cross-entropy loss calculation method. The terminal determines the third loss function according to the prediction offset corresponding to each pixel point with the second prediction position label being a head pixel point or a tail pixel point in the sample image, and the sample offset corresponding to each pixel point with the second sample position label being a head pixel point or a tail pixel point in the sample image according to the Smooth L1 loss calculation method. The terminal performs weighted summation on the first loss function, the second loss function and the third loss function according to the preset weight to obtain the target loss function.

[0095] It can be understood that the terminal calculates the first loss function and the second loss function according to the existing balanced cross-entropy loss calculation method, and calculates the third loss function according to the existing smooth L1 regularization loss calculation method, which will not be repeated here.

[0096] In one embodiment, the terminal regresses two predicted top points of the head region of the corresponding shelf line based on the offset and position coordinates of each pixel point with the second prediction position label as the head pixel point in the sample image in the above manner, and regresses two predicted top points of the tail region of the corresponding shelf line based on the offset and position coordinates of each pixel point with the second prediction position label as the tail pixel point in the sample image in the above manner. Further, the terminal determines the third loss function according to the Smooth L1 loss calculation method based on the four predicted top points and the four sample top points of each shelf line in the sample image. It can be understood that the four sample top points corresponding to each shelf line in the sample image can be obtained by manual labeling during the training sample collection process, or can be obtained by the terminal based on the offset and position coordinates of each pixel point with the second sample position label as the head pixel point or the tail pixel point in the sample image in the above manner.

[0097] Thus, the pixel point loss of the balanced cross-entropy is used to optimize whether it is a pixel point inside the shelf line, the head and tail region loss of the balanced cross-entropy is used to optimize whether it is a pixel point in the head and tail region, and the Smooth L1 loss is used to optimize the position coordinates of the four top points of each shelf line obtained by regression.

[0098] In one embodiment, the terminal determines the target loss function according to the following mapping relationship.

[0099] L = L pixel + λ vertex * L vertex + λ geo * L geo

[0100] L represents the target loss function, L pixel represents the first loss function, L vertex and λ vertex respectively represent the second loss function and the preset weight corresponding to the second loss function, L geo and λ geo respectively represent the third loss function and the preset weight corresponding to the third loss function.

[0101] In the above embodiments, whether it is an internal pixel point of the shelf line, whether it is a head or tail pixel point, and the offset of each head or tail pixel point are determined respectively to determine the corresponding loss function, and the target loss function is determined according to the three loss functions based on the preset weight to improve the training efficiency of the model and the detection accuracy of the target detection model obtained by training.

[0102] In one embodiment, the obtaining step of the training sample set comprises: obtaining a sample image and a sample shelf line vertex corresponding to each sample shelf line in each sample image; preprocessing the sample shelf line vertex corresponding to each sample shelf line to obtain a corresponding sample head region and a sample tail region; determining a corresponding sample feature set based on the sample head region and the sample tail region corresponding to each sample shelf line in each sample image; and obtaining a training sample set according to the sample image and the sample feature set corresponding to each sample image.

[0103] Specifically, the terminal obtains a plurality of sample images, and manually labels four shelf line vertices corresponding to each sample shelf line in each sample image by manual marking to obtain a sample shelf line vertex corresponding to each sample shelf line in each sample image. For each sample shelf line in each sample image, the terminal determines the height, top width and bottom width of the head region of the sample shelf line according to the two sample shelf line vertices corresponding to the sample shelf line and located in the head region of the sample shelf line, and determines the sample head region of the sample shelf line in the sample image based on the determined height, top width and bottom width. Correspondingly, the terminal determines the sample tail region of the sample shelf line in the sample image according to the two sample shelf line vertices corresponding to the sample shelf line and located in the tail region of the sample shelf line in a similar manner.

[0104] Further, for each sample image, the terminal can determine a sample shelf line region corresponding to each sample shelf line in the sample image according to the sample shelf line vertex corresponding to each sample shelf line in the sample image. Therefore, the terminal can determine a first sample position label and a second sample position label corresponding to each pixel point in the sample image according to the sample shelf line region, the sample head region and the sample tail region corresponding to each sample shelf line in each sample image, determine a sample offset for each pixel point in the sample image that is a head pixel point or a tail pixel point, and obtain a sample feature set corresponding to the sample image according to the first sample position label, the second sample position label and the sample offset corresponding to each pixel point in the sample image, and obtain a training sample set according to each sample image and the corresponding sample feature set.

[0105] In one embodiment, for each sample shelf line in each sample image, the terminal designates the four vertices of the sample shelf line as the top-left, bottom-left, top-right, and bottom-right points. The distance between the top-left and bottom-left points is determined as the height of the head region of the shelf line. The terminal compares the product of the top width of the sample shelf line and a preset ratio with a threshold value for the width of the head and tail regions of the shelf line, and determines the smaller of the product and the threshold value as the top width of the head region of the sample shelf line. Similarly, the terminal compares the product of the bottom width of the sample shelf line and a preset ratio with a threshold value for the width of the head and tail regions of the shelf line, and determines the smaller of the product and the threshold value as the bottom width of the head region of the sample shelf line. The top width of the sample shelf line is the distance between the top-left and top-right points, and the bottom width is the distance between the bottom-left and bottom-right points. The preset ratio can be customized, for example, 0.4. The shelf line start and end area width threshold refers to the maximum limit of the width of the shelf line start and end area. It can be customized. For example, for a 512*512 target image or sample image, the shelf line start and end area width threshold can be set to 48.

[0106] Figure 2 This is a schematic diagram illustrating the determination of the head and tail regions based on the vertices of the shelving line in one embodiment. For example... Figure 2 As shown, the shelf line includes four vertices: top left, bottom left, top right, and bottom right. Based on the top width of the shelf line, a preset ratio, and a threshold width for the head and tail regions, the top width of the head region is determined as the product of the top width and the preset ratio. Similarly, the bottom width of the head region is the product of the bottom width and the preset ratio. Therefore, based on the bottom and top widths of the head region, the head region of the shelf line can be determined, and each pixel within this head region is identified as a head pixel, meaning the first position label corresponding to each pixel within the head region is also identified as a head pixel. Thus, based on each head pixel, the two shelf line vertices within the head region can be regressed. Similarly, the tail region of the shelf line can be determined in the same way, and each pixel within the tail region is identified as a tail pixel. Thus, based on each tail pixel, the two shelf line vertices within the tail region can be regressed.

[0107] It can be understood that for a longer length shelf line, by selecting a certain width of the head region and the tail region at the head and tail respectively, each pixel point based on the head region can accurately regress the two shelf line vertices of the head region, and each pixel point based on the tail region can accurately regress the two shelf line vertices of the tail region. Therefore, by setting the preset proportion and the shelf line head and tail region width threshold, the terminal can dynamically determine the tail region and the head region which can be used to accurately regress the four shelf line vertices according to the top width and the bottom width of the shelf line, and the set preset proportion and the shelf line head and tail region width threshold.

[0108] In one embodiment, the characteristic of the shelf line is that the left and right distance is longer and the up and down distance is shorter, so by selecting the regions at the left and right ends of the shelf line as the head region and the tail region respectively, and using the pixel points in the head region to predict the two shelf line vertices of the shelf line in the head region, and using the pixel points in the tail region to predict the two shelf line vertices of the shelf line in the tail region. Compared with predicting the four shelf line vertices of the corresponding shelf line based on all pixel points in the entire shelf line region, this kind of predicting the two shelf line vertices of the corresponding region based on the pixel points in the head region and the tail region can improve the positioning accuracy of the shelf line vertex, so as to improve the detection accuracy of the shelf line based on the positioning of the four shelf line vertices.

[0109] As shown in Figure 3 , in one embodiment, a shelf line detection method is provided, which specifically comprises the following steps:

[0110] Step 302, acquiring a target image.

[0111] Step 304, determining the first position label and the second position label corresponding to each pixel point in the target image, and the offset corresponding to each pixel point whose second position label is a head pixel point or a tail pixel point in the target image by using the trained target detection model; the first position label is used to represent whether the corresponding pixel point is a shelf line internal pixel point, and the second position label is used to represent whether the corresponding pixel point is a head pixel point or a tail pixel point.

[0112] Step 306, by traversing the pixel points in the target image, clustering the first position label in each row as a shelf line internal pixel point and the adjacent pixel points to obtain a shelf line pixel point subset, and merging the shelf line pixel point subsets which are in adjacent two rows and have at least one pixel point with the same horizontal coordinate to obtain at least one shelf line pixel point set.

[0113] Step 308, in each shelf line pixel point set, the head pixel point set and the corresponding tail pixel point set are determined based on the second position label corresponding to each pixel point.

[0114] In step 310, two shelf line top points in the head region of the corresponding shelf line are determined based on the offset of each pixel point in the head pixel point set.

[0115] In step 312, two shelf line top points in the tail region of the corresponding shelf line are determined based on the offset of each pixel point in the tail pixel point set.

[0116] In step 314, the corresponding shelf line is located in the target image based on the shelf line top points in the head region and the shelf line top points in the tail region of the corresponding shelf line.

[0117] In the above embodiment, by using the trained target detection model, the first position label and the second position label corresponding to each pixel point in the target image, and the offset of each pixel point with the second position label being a head pixel point or a tail pixel point are determined. Based on the first position label, the shelf line pixel point set corresponding to each shelf line in the target image is determined. In each shelf line pixel point set, the head region and the tail region of the corresponding shelf line are determined based on the second position label. Based on the offset of each pixel point in the head region and the tail region, four shelf line top points of the corresponding shelf line are determined, so as to locate the corresponding shelf line in the target image based on the shelf line top points. This kind of shelf line detection method has the effects of high detection accuracy and stability, and strong generalization ability, and can be applied to accurate detection of shelf lines in various detection scenes, so as to further estimate the pose of the shelf line based on the accurate detection of the shelf line, or correct the shelf line according to the actual situation.

[0118] It can be understood that by using the trained target detection model, the first position label, the second position label and the offset corresponding to each pixel point in the target image can be quickly and accurately determined, so as to improve the detection efficiency and accuracy when the shelf line is detected based on the first position label, the second position label and the offset. Moreover, the image size of the input feature of the target detection model can be dynamically adjusted according to the demand, so as to adapt to the detection speed requirement of different scenes.

[0119] Figure 4 A schematic diagram of the principle of detecting the target image based on the target detection model in one embodiment. Taking the machine learning algorithm involved in training the target detection model as a full convolutional network for example, as shown in FIG. 4. Figure 4As shown, the target detection model internally includes a full convolutional network, a feature fusion layer, and an output layer in sequence. After inputting a target image as an input feature into the target detection model, the target detection model extracts features of the target image through the full convolutional network. The full convolutional network includes multiple feature layers. Each feature layer can be understood as a feature extraction stage. Taking the full convolutional network with Efficientnet_b0 as an example, the full convolutional network includes 9 feature layers. The feature map sizes output by different feature layers are different. The feature map sizes output by two adjacent feature layers are in a two-fold relationship. The receptive fields of the pixel points in the feature maps are also different. Figure 4 Only five feature layers, i.e., feature layer 1, feature layer 3, feature layer 5, feature layer 7, and feature layer 8, are shown. The feature map sizes output by feature layer 3, feature layer 5, feature layer 7, and feature layer 8 are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the size of the target image, respectively. That is, the feature map sizes output by two adjacent feature layers in the four feature layers are in a two-fold relationship. Therefore, the feature maps output by multiple feature layers are selected from the four feature layers for multi-size feature fusion, such as Figure 4 As shown, the feature maps output by the four feature layers are sequentially subjected to multi-size feature fusion to take into account the detection of different sizes of shelf lines.

[0120] The target detection model performs upward fusion on the feature maps output by the four feature layers at the feature fusion layer. As shown in Figure 4As shown, the feature map f1 output by the feature layer 8 is taken as the initial fusion feature map h1 of the feature fusion layer, the fusion feature map h1 is fused with the feature map f2 output by the feature layer 7 to obtain a fusion feature map h2, the fusion feature map h2 is fused with the feature map f3 output by the feature layer 5 to obtain a fusion feature map h3, the fusion feature map h3 is fused with the feature map f4 output by the feature layer 3 to obtain a fusion feature map h4, and the fusion feature map h4 is convolved with a convolution kernel with a size of 3*3 and a channel number of 32 to obtain the feature map finally output by the feature fusion layer. The target detection model obtains a feature set corresponding to the target image according to the feature map finally output by the feature fusion layer at the output layer, and the feature set includes a first position label representing whether it is a pixel point inside the shelf line, a second position label representing whether it is a head / tail pixel point, and four offsets corresponding to the head / tail pixel point. The step of fusing the fusion feature map with the feature map includes upsampling the fusion feature map, expanding the size of the fusion feature map to be consistent with the size of the feature map to be fused through upsampling, fusing the upsampled fusion feature map with the feature map (concat), adding the channel numbers of the fusion feature map and the feature map, and sequentially convolving the fused feature map with a convolution kernel with a size of 1*1 and 3*3. The convolution with the convolution kernel with a size of 1*1 and 3*3 can reduce the channel number and the amount of calculation. Different convolution kernels with different channel numbers are selected for the feature maps output by different feature layers, such as Figure 4 As shown, the channel numbers of the selected convolution kernels are 128, 64, and 32 in turn.

[0121] Figure 5 is an example of the shelf line detection effect achieved based on the shelf line detection method in an embodiment. For the target image shown in Figure 5 label 502, the three shelf lines shown in label 504 can be detected from the target image in the above manner, wherein the label 506 corresponds to a commodity placed on the shelf line. Thus, the shelf line can be accurately detected from the target image in the above manner. It should be noted that, Figure 5 The target image, the shelf line in the target image, and the commodity on the shelf line shown in

[0122] It should be understood that, although Figure 1 and Figure 3 the steps in the flowcharts are displayed in turn according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, Figure 1 and Figure 3At least one of the steps in the method can comprise a plurality of steps or stages which are not necessarily performed at the same time but can be performed at different times and the order of the steps or stages is not necessarily sequential but can be performed in an order which is round-robin or alternating with other steps or steps or stages in other steps.

[0123] In one embodiment, as shown in FIG. 6, there is provided a shelf line detection apparatus 600, comprising: an acquisition module 601, a label determination module 602, a first set determination module 603, a second set determination module 604, and a detection module 605, wherein: Figure 6

[0124] The acquisition module 601 is configured to acquire a target image.

[0125] The label determination module 602 is configured to determine a first position label and a second position label corresponding to each pixel point in the target image; the first position label is used to represent whether the corresponding pixel point is a shelf line internal pixel point, and the second position label is used to represent whether the corresponding pixel point is a head pixel point or a tail pixel point.

[0126] The first set determination module 603 is configured to determine at least one shelf line pixel point set based on the first position label corresponding to each pixel point in the target image.

[0127] The second set determination module 604 is configured to determine a head pixel point set and a corresponding tail pixel point set in each shelf line pixel point set based on the second position label corresponding to each pixel point.

[0128] The detection module 605 is configured to determine four shelf line vertices corresponding to a corresponding shelf line in the target image based on each head pixel point set and the corresponding tail pixel point set, and to locate the shelf line from the target image based on the shelf line vertices.

[0129] In one embodiment, the first set determination module 603 is further configured to obtain a shelf line pixel point subset by traversing the pixel points in the target image, clustering the first position label in each row as a shelf line internal pixel point and adjacent pixel points, and merging the shelf line pixel point subsets which are located in adjacent two rows and have at least one pixel point with the same horizontal coordinate, to obtain at least one shelf line pixel point set.

[0130] ​In an embodiment, when the second position label of the pixel point in the target image is a head pixel point, the pixel point corresponds to an offset; when the second position label of the pixel point in the target image is a tail pixel point, the pixel point corresponds to an offset; the detection module 605 is further configured to determine two shelf line top points in the head region of the corresponding shelf line based on the offset corresponding to each pixel point in each head pixel point set; determine two shelf line top points in the tail region of the corresponding shelf line based on the offset corresponding to each pixel point in each tail pixel point set; and locate the corresponding shelf line from the target image based on the shelf line top point in each head region of the shelf line and the shelf line top point in the corresponding tail region of the shelf line.

[0131] In an embodiment, the above shelf line detection device 600 further comprises a training model.

[0132] The training model is configured to obtain a training sample set; the training sample set comprises a sample image and a sample feature set corresponding to the sample image; the sample feature set comprises a first sample position label and a second sample position label corresponding to each pixel point in the sample image, and a sample offset corresponding to a pixel point whose second sample position label is a head pixel point or a tail pixel point; perform model training based on the training sample set to obtain a trained target detection model.

[0133] In an embodiment, the training model is further configured to detect the sample image by using the target detection model to be trained to obtain a predicted feature set corresponding to the sample image; determine a target loss function based on the sample feature set and the predicted feature set, and adjust the model parameters of the target detection model to be trained based on the target loss function; perform the step of detecting the sample image by using the target detection model to be trained to obtain the predicted feature set corresponding to the sample image until iteration is stopped to obtain the trained target detection model.

[0134] In an embodiment, the predicted feature set comprises a first predicted position label and a second predicted position label corresponding to each pixel point in the sample image, and a predicted offset corresponding to a pixel point whose second predicted position label is a head pixel point or a tail pixel point; the training model is further configured to determine a first loss function based on the first predicted position label corresponding to each pixel point in the sample image and the first sample position label, determine a second loss function based on the second predicted position label corresponding to each pixel point in the sample image and the second sample position label, and determine a third loss function based on the predicted offset corresponding to each pixel point whose second predicted position label is a head pixel point or a tail pixel point in the sample image, and the sample offset corresponding to each pixel point whose second sample position label is a head pixel point or a tail pixel point; obtain a corresponding target loss function according to the first loss function, the second loss function and the third loss function according to a preset weight.

[0135] In an embodiment, the training model is further configured to obtain a sample image, and a sample shelf line vertex corresponding to each sample shelf line in the sample image; perform preprocessing on the sample shelf line vertex corresponding to each sample shelf line to obtain a corresponding sample head region and a sample tail region; determine a corresponding sample feature set based on the sample head region and the sample tail region corresponding to each sample shelf line in the sample image; and obtain a training sample set according to the sample image and the sample feature set corresponding to each sample image.

[0136] The specific limitations of the shelf line detection device can refer to the limitations of the shelf line detection method described above, and will not be repeated here. Each module in the above shelf line detection device can be realized by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to the above modules.

[0137] In an embodiment, a computer device is provided, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 7 The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a shelf line detection method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0138] Those skilled in the art can understand that Figure 7 The structure shown in the above

[0139] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory has stored therein a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0140] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program, and the computer program, when executed by a processor, implements the steps in the above method embodiments.

[0141] A person of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0142] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0143] The above embodiments only express several implementation manners of the present application, and the description is specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the patent of the present application should be subject to the appended claims.

Claims

1. A method for detecting shelving lines, characterized in that, The method includes: Acquire the target image; Using a pre-trained target detection model, a first position label and a second position label are determined for each pixel in the target image; the first position label is used to characterize whether the corresponding pixel is a pixel inside the shelf line, and the second position label is used to characterize whether the corresponding pixel is a head pixel or a tail pixel. Based on the first location label corresponding to each pixel in the target image, at least one set of shelf line pixels is determined. In each set of pixels along a shelf line, a set of head pixels and a set of corresponding tail pixels are determined based on the second position label corresponding to each pixel. Specifically, when the second position label in the target image is a head pixel, the pixel has an offset; when the second position label in the target image is a tail pixel, the pixel has an offset. Based on the offset of each pixel in each head pixel set, determine the two shelf line vertices in the head region of the corresponding shelf line; based on the offset of each pixel in each tail pixel set, determine the two shelf line vertices in the tail region of the corresponding shelf line. The shelf line is located from the target image based on the vertex of the shelf line.

2. The method according to claim 1, characterized in that, The step of determining at least one set of shelf line pixels based on the first position label corresponding to each pixel in the target image includes: By traversing the pixels in the target image, the pixels in each row that are labeled as being inside the shelf line and are adjacent to each other are clustered to obtain a subset of shelf line pixels. The subsets of shelf line pixels that are located in two adjacent rows and have at least one pixel with the same horizontal coordinate are merged to obtain at least one set of shelf line pixels.

3. The method according to claim 1, characterized in that, The step of locating the shelf line from the target image based on the shelf line vertices includes: The corresponding shelf line is located from the target image based on the shelf line vertex in the head region of each shelf line and the shelf line vertex in the corresponding tail region of each shelf line.

4. The method according to claim 3, characterized in that, The training steps of the target detection model include: Obtain a training sample set; the training sample set includes a sample image and a sample feature set corresponding to the sample image; the sample feature set includes a first sample position label and a second sample position label corresponding to each pixel in the sample image, and a sample offset corresponding to a pixel whose second sample position label is a head pixel or a tail pixel; The model is trained based on the training sample set to obtain a trained target detection model.

5. The method according to claim 4, characterized in that, The training steps of the object detection model also include: The sample image is detected by the target detection model to be trained, and the predicted feature set corresponding to the sample image is obtained. The target loss function is determined based on the sample feature set and the predicted feature set, and the model parameters of the target detection model to be trained are adjusted in reverse based on the target loss function. The process involves performing the step of detecting the sample image using the target detection model to be trained, and obtaining the predicted feature set corresponding to the sample image, until the iteration stops, resulting in a trained target detection model.

6. The method according to claim 5, characterized in that, The prediction feature set includes a first prediction position label and a second prediction position label corresponding to each pixel in the sample image, and a prediction offset corresponding to a pixel whose second prediction position label is a head pixel or a tail pixel. The step of determining the target loss function based on the sample feature set and the predicted feature set includes: A first loss function is determined based on the first predicted location label and the first sample location label corresponding to each pixel in the sample image. A second loss function is determined based on the second predicted location label and the second sample location label corresponding to each pixel in the sample image. A third loss function is determined based on the predicted offset corresponding to each pixel in the sample image whose second predicted location label is a head pixel or a tail pixel, and the sample offset corresponding to each pixel whose second sample location label is a head pixel or a tail pixel. Based on the first loss function, the second loss function, and the third loss function, a corresponding target loss function is obtained according to preset weights.

7. The method according to any one of claims 4 to 6, characterized in that, The steps for obtaining the training sample set include: Obtain sample images, and the vertex of each sample shelf line in each sample image; Preprocess the vertex of each sample shelf line to obtain the corresponding sample head region and sample tail region; Based on the sample head region and sample tail region corresponding to each sample shelf line in each sample image, the corresponding sample feature set is determined; The training sample set is obtained based on the sample images and the sample feature set corresponding to each sample image.

8. A shelf line detection device, characterized in that, The device includes: The acquisition module is used to acquire the target image; The label determination module is used to determine a first position label and a second position label corresponding to each pixel in the target image; the first position label is used to characterize whether the corresponding pixel is a pixel inside the shelf line, and the second position label is used to characterize whether the corresponding pixel is a head pixel or a tail pixel. The first set determination module is used to determine at least one set of shelf line pixels based on the first position label corresponding to each pixel in the target image. The second set determination module is used to determine the head pixel set and the corresponding tail pixel set in each shelf line pixel set based on the second position label corresponding to each pixel, wherein the pixel has an offset when the second position label in the target image is a head pixel; and the pixel has an offset when the second position label in the target image is a tail pixel. The detection module is used to determine two shelf line vertices in the head region of the corresponding shelf line based on the offset of each pixel in each head pixel set; determine two shelf line vertices in the tail region of the corresponding shelf line based on the offset of each pixel in each tail pixel set; and locate the shelf line from the target image based on the shelf line vertices.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images