Method, device, computer device, and storage medium for detecting object height

Abstract

The present application relates to a method, device, computer device, and storage medium for detecting the height of an object. The method includes: obtaining an image of a container to be detected; determining first position information of the top contour of the container and second position information of the edge of a target object in the image of the container to be detected; performing perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix, where the perspective transformation matrix is used to convert the image of the container to be detected into an image in a first viewing angle, to obtain a target container image; determining target position information corresponding to the second position information in the target container image; and generating the height of the target object according to the comparison result between the target position information and a plurality of reference scales. By using this method, the height of the target object can be generated based on the comparison result between the target position information corresponding to the edge of the target object in the target container image in the first viewing angle and the reference scales, thereby improving the detection accuracy of the object height.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a method, device, computer device, storage medium, and computer program product for detecting the height of an object. Background Art

[0002] With the development of image processing technology, image measurement technology has emerged. Users can measure and estimate the features of a target detection object in a to-be-detected image through image measurement technology to determine information such as the position, geometric dimensions, or appearance of the target detection object. Taking the measurement of an object in a trash can in the environmental protection field as an example, users can process the internal image of the trash can through image measurement technology to determine the stacking height of the object inside the trash can, so as to facilitate subsequent intelligent management of the trash can.

[0003] In traditional technology, target detection processing can be performed on the image of the trash inside the bin collected by a monocular camera to determine the pixel coordinates of the trash area in the trash image inside the bin. Based on the relative position relationship between the monocular camera and the bin, arithmetic processing is performed on the pixel coordinates of the trash area to generate the stacking height of the trash inside the bin. However, when using the object height detection method in traditional technology, due to the relative position relationship between the monocular camera and the bin being prone to errors, the detection accuracy of the object height is relatively low. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for detecting the height of an object with high detection accuracy.

[0005] In a first aspect, the present application provides a method for detecting the height of an object. The method includes:

[0006] Obtain an image of a to-be-detected container;

[0007] Determine first position information of the top contour of the container and second position information of the edge of the target object inside the container in the to-be-detected container image;

[0008] Perform perspective transformation processing on the to-be-detected container image according to the first position information and a preset perspective transformation matrix, to obtain a target container image, where the perspective transformation matrix is used to convert the to-be-detected container image into an image in a first perspective;

[0009] Determine target position information corresponding to the second position information in the target container image;

[0010] Generate the height of the target object according to the comparison result between the target position information and a plurality of reference scales.

[0011] In one embodiment, the obtaining of the image of the container to be detected includes:

[0012] In response to an image acquisition operation, an image acquisition page is displayed, and an image mask is displayed on the image acquisition page. The image mask is generated according to a first standard image, and the first standard image is obtained by photographing a container sample from a second perspective;

[0013] The image of the container to be detected that matches the image mask is obtained through the image acquisition page.

[0014] In one embodiment, the generation method of the perspective transformation matrix includes:

[0015] The first standard image is detected to obtain third position information of the container top contour in the first standard image;

[0016] The second standard image is detected to obtain fourth position information of the container top contour in the second standard image. The second standard image is obtained by photographing the container sample from the first perspective;

[0017] The perspective transformation matrix is generated according to the third position information and the fourth position information.

[0018] In one embodiment, the performing perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix to obtain a target container image includes:

[0019] Performing perspective transformation processing on the image of the container to be detected according to the first position information and the preset perspective transformation matrix to obtain the image of the container to be detected after perspective transformation processing;

[0020] According to the fourth position information, the image height of the second standard image is determined;

[0021] The image of the container to be detected after perspective transformation processing is scaled according to the image height to obtain the target container image.

[0022] In one embodiment, the determining of the first position information of the container top contour in the image of the container to be detected includes:

[0023] The image of the container to be detected is input into a detection model to obtain the first position information. The detection model is trained according to a sample container image and a target label corresponding to the sample container image. The target label is obtained by performing Gaussian blur processing on the original label of the sample container image, and the original label is obtained by labeling multiple boundary points of the container top contour in the sample container image in a preset order.

[0024] In one embodiment, determining the second position information of the edge of the target object in the container includes:

[0025] Performing image segmentation processing on the image of the container to be detected to obtain the image inside the container;

[0026] Performing line detection processing on the image inside the container to generate a line detection result of the image inside the container;

[0027] Performing clustering processing on the line detection result to obtain a line segment clustering result, and determining the area where the target object is located according to the line segment clustering result;

[0028] Performing edge detection on the area where the target object is located to determine the second position information.

[0029] In a second aspect, the present application further provides a device for detecting the height of an object, characterized in that the device includes:

[0030] An image acquisition module, configured to acquire an image of the container to be detected;

[0031] A first determination module, configured to determine the first position information of the top contour of the container in the image of the container to be detected, and the second position information of the edge of the target object in the container;

[0032] A perspective transformation module, configured to perform perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix to obtain a target container image, where the perspective transformation matrix is used to convert the image of the container to be detected into an image in a first perspective;

[0033] A second determination module, configured to determine the target position information corresponding to the second position information in the target container image;

[0034] A height generation module, configured to generate the height of the target object according to the target position information.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the method for detecting the height of an object according to any one of the embodiments in the first aspect above.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method for detecting the height of an object according to any one of the embodiments in the first aspect above.

[0037] Fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program which, when executed by a processor, implements the method for detecting the height of an object according to any one of the embodiments in the first aspect above.

[0038] The above method, device, computer device, storage medium and computer program product for detecting the height of an object, by acquiring an image of a container to be detected, determining first position information of the top contour of the container in the image of the container to be detected and second position information of the edge of the target object in the container, performing perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix to obtain a target container image from a first perspective, determining target position information corresponding to the second position information in the target container image, and generating the height of the target object according to the comparison result between the target position information and a plurality of reference scales, can directly perform perspective transformation processing on the image of the container to be detected by using the position information of the top contour of the container in the image of the container to be detected, determine, based on the target container image from the first perspective obtained after perspective transformation, the target position information corresponding to the edge of the target object in the container in the image of the container to be detected, compare the target position information with a plurality of reference scales to generate the height of the target object in the container, thereby improving the detection accuracy of the object height. In addition, when using the method for detecting the height of an object provided by the present application, since there is no need to process pixel positions by using the relative position relationship between the image acquisition device and the container, the detection process of the object height can be simplified, thereby improving the detection efficiency of the object height. Description of the Drawings

[0039] Figure 1 It is a schematic flowchart of the method for detecting the height of an object in one embodiment;

[0040] Figure 2 It is a schematic flowchart of the step of generating a perspective transformation matrix in one embodiment;

[0041] Figure 3 It is a schematic flowchart of the step of generating a target container image in one embodiment;

[0042] Figure 4 It is a schematic flowchart of the step of determining the second position information in one embodiment;

[0043] Figure 5 It is a schematic flowchart of the method for detecting the height of an object in another embodiment;

[0044] Figure 5a It is a schematic diagram of the step of generating an image mask in one embodiment;

[0045] Figure 5b It is a schematic diagram of the step of acquiring an image of a container to be detected in one embodiment;

[0046] Figure 5c Schematic diagram of the first position information determination step in an embodiment;

[0047] Figure 5d Schematic diagram of the second position information determination step in an embodiment;

[0048] Figure 5e Schematic diagram of the target container image generation step in an embodiment;

[0049] Figure 5f Schematic diagram of the perspective transformation matrix generation step in an embodiment;

[0050] Figure 5g Schematic diagram of the target object height generation step in an embodiment;

[0051] Figure 6 Structural block diagram of the detection device for the object height in an embodiment;

[0052] Figure 7 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The detection method for the object height provided by the embodiment of the present application can be applied to a computer device. Among them, the computer device can be a terminal, a server, or a system including a terminal and a server. Specifically, the computer device stores an object height generation logic and a preset perspective transformation matrix. The object height generation logic includes a plurality of reference scales, and the perspective transformation matrix is used to convert the image of the container to be detected into an image in the first perspective. The computer device acquires the image of the container to be detected, performs target detection on the top contour of the container in the image of the container to be detected, and determines the first position information of the top contour of the container. Edge detection is performed on the target object in the container in the image of the container to be detected to determine the second position information of the edge of the target object. The perspective transformation process is performed on the image of the container to be detected according to the first position information and the perspective transformation matrix to obtain a target container image in the first perspective. The target position information corresponding to the second position information in the target container image is determined. The object height generation logic is used to process the target position information, compare the target position information with a plurality of reference scales, and generate the height of the target object according to the comparison result.

[0055] Among them, the terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart TVs, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0056] In an embodiment disclosed in the present application, as Figure 1 shown, a method for detecting the height of an object is provided. This method is applied to the scenario of detecting the height of an object in a container. Taking the application of this method to a terminal as an example, the method includes the following steps:

[0057] Step S102, obtain an image of the container to be detected.

[0058] Among them, the image of the container to be detected can be an image captured for an open container, and the open container can include a target object. In one example, the image of the container to be detected can be an image collected for an open trash can, and the target object can be displayed in the open area of the trash can.

[0059] Specifically, the terminal responds to a detection request for the object height and obtains an image of the container to be detected. Among them, the detection request for the object height can be automatically triggered by the terminal. For example, after the terminal detects the image of the container to be detected, it automatically triggers a detection request for the object height of the image of the container to be detected; it can also be manually triggered by the user as needed. For example, the user clicks a corresponding detection button on the terminal interface to trigger a detection request for the object height and obtain an image of the container to be detected in the local database or server.

[0060] Step S104, determine the first position information of the top contour of the container in the image of the container to be detected, and the second position information of the edge of the target object in the container.

[0061] Among them, the first position information can be used to characterize the positions of the pixel points on the top contour of the container. In one example, the first position information can include the coordinates of each pixel point on the top contour of the container. In another example, the first position information can include the coordinates of multiple pixel points at fixed positions on the top contour of the container, such as the coordinates of the pixel points at the four corners of the top contour of the container.

[0062] The second position information can be used to characterize the positions of the pixel points at the edge of the target object. In one example, the second position information can include the coordinates of each pixel point at the edge of the target object. In another example, the second position information can include the coordinates of multiple pixel points at a preset edge of the target object, such as the coordinates of multiple pixel points at the upper edge of the target object.

[0063] Specifically, the terminal performs object detection on the container top contour in the container image to be detected, and determines the first position information of the container top contour. Edge detection is performed on the target object inside the container in the container image to be detected to determine the second position information of the target object edge. Among them, the first position information can be, but is not limited to, determined by any one of multiple object detection models such as the Faster R-CNN model (Faster Region-Convolutional Neural Networks), the SSD model (Single Shot MultiBox Detector), and the YOLO model (You Only LookOnce, a small and fast object detection model). The second position information can be determined by any one of multiple edge detection operators such as the Roberts Cross gradient operator, the Sobel operator, the Canny operator, and the Laplacian operator.

[0064] Step S106: Perform perspective transformation processing on the container image to be detected according to the first position information and the preset perspective transformation matrix to obtain the target container image.

[0065] Among them, the perspective transformation matrix can be used to convert the container image to be detected into an image in the first perspective.

[0066] The first perspective can be used to represent the perspective of the image acquisition device when collecting an image of the container from above the container. In one example, the first perspective can be the top-down perspective of the image acquisition device when collecting an image of the container directly from above the container.

[0067] Specifically, the perspective transformation matrix is stored in the terminal. The internal area of the container in the container image to be detected that needs to be subjected to perspective transformation is determined according to the first position information. The position information of each pixel point in the internal area of the container is obtained, and the perspective transformation matrix is used to process the position information of each pixel point in the internal area of the container to generate the position information of each pixel point after transformation. The target container image is generated according to the position information of each pixel point after transformation in the internal area of the container.

[0068] Step S108: Determine the target position information corresponding to the second position information in the target container image.

[0069] Specifically, the terminal performs arithmetic processing on the second position information of the target object edge using the perspective transformation matrix to generate the target position information corresponding to the second position information in the target container image, and uses the target position information as the position information of the target object edge in the target container image.

[0070] Step S110: Generate the height of the target object according to the comparison result between the target position information and multiple reference scales.

[0071] Specifically, the height generation logic of the object is stored in the terminal, and multiple reference scales are included in the height generation logic of the object. The height generation logic of the object is used to process the target position information of the target object, compare the target position information with multiple reference scales, and generate the height of the target object according to the comparison result. Among them, the height generation logic of the object can be to compare the target position information with the reference position information of multiple reference scales in the horizontal direction, and generate the height of the target object according to the comparison result between the position information; it can also be to determine the pixel size of the edge of the target object according to the target position information, compare the pixel size with the reference pixel sizes corresponding to multiple reference scales, and generate the height of the target object according to the comparison result between the pixel sizes; it can also be to determine the distance between the edge of the target object and the top contour of the container in the target container image according to the target position information of the edge of the target object and the position information of the top contour of the container, compare the distance with the reference distances corresponding to multiple reference scales, and generate the height of the target object according to the comparison result between the distances.

[0072] In an example, the target position information of the edge of the target object obtained by the terminal is (3, 3), the first reference position of the first reference scale in the horizontal direction is (1, 1), the second reference position of the second reference scale in the horizontal direction is (1, 2), and the third reference position of the third reference scale in the horizontal direction is (1, 3). The target position information is compared with the reference positions of the three reference scales in the horizontal direction in turn, and it is determined that the matching degree between the target position information and the reference position of the third reference scale in the horizontal direction is the highest, and the object height corresponding to the third reference scale is used as the height of the target object.

[0073] In the above method for detecting the height of an object, by obtaining an image of the container to be detected, determining the first position information of the top contour of the container in the image of the container to be detected and the second position information of the edge of the target object in the container, performing a perspective transformation process on the image of the container to be detected according to the first position information and a preset perspective transformation matrix to obtain an image of the target container from the first perspective, determining the target position information corresponding to the second position information in the image of the target container, and generating the height of the target object based on the comparison result between the target position information and multiple reference scales, it is possible to directly use the position information of the top contour of the container in the image of the container to be detected to perform a perspective transformation process on the image of the container to be detected. Based on the image of the target container from the first perspective obtained after the perspective transformation, determining the target position information corresponding to the edge of the target object in the container in the image of the container to be detected, comparing the target position information with multiple reference scales to generate the height of the target object in the container, thereby improving the detection accuracy of the object height. In addition, when using the method for detecting the height of an object provided in this application, since there is no need to process the pixel positions based on the relative position relationship between the image acquisition device and the container, the detection process of the object height can be simplified, thereby improving the detection efficiency of the object height.

[0074] In an embodiment disclosed in this application, step S102, obtaining an image of the container to be detected, includes: in response to an image acquisition operation, displaying an image acquisition page, displaying an image mask on the image acquisition page, and obtaining an image of the container to be detected that matches the image mask through the image acquisition page.

[0075] Among them, the image mask can be a container contour template generated according to a first standard image, and can be used to instruct the user to acquire an image according to the image acquisition angle corresponding to the image mask.

[0076] The first standard image can be an image obtained by photographing a container sample from a second perspective. In one example, the container sample can be an open empty container.

[0077] The second perspective can be used to represent the perspective of the image acquisition device when acquiring an image of the container sample from the front of the container sample. In one example, the image acquisition device can acquire a complete top contour of the container sample from the second perspective.

[0078] Specifically, an image mask is pre-stored in the terminal. In response to an image acquisition operation triggered by the user, an image acquisition page is displayed, and the image mask is displayed on the image acquisition page. A container image is obtained through the image acquisition page, edge detection is performed on the container image to determine the container contour in the container image, and the obtained container contour is compared with the image mask to determine the similarity between the container image and the image mask. When the similarity between the container image and the image mask is greater than a threshold, it is determined that the container image matches the image mask, and the container image is used as the container image to be detected that matches the image mask. Among them, the container image can be an image captured in real time by an image capture component deployed on the terminal, or an image obtained by the terminal from a local database or a server.

[0079] In this embodiment, by displaying the image mask on the image acquisition page and obtaining the container image to be detected that matches the image mask through the image acquisition page, the image acquisition angle of the container image to be detected can be unified, so that the container top contour required for object height detection is included in the container image to be detected, thereby improving the processing efficiency of the subsequent container image to be detected.

[0080] In an embodiment disclosed in the present application, as Figure 2 shown, the generation method of the perspective transformation matrix includes:

[0081] Step S202, detecting the first standard image to obtain the third position information of the container top contour in the first standard image.

[0082] Step S204, detecting the second standard image to obtain the fourth position information of the container top contour in the second standard image.

[0083] Step S206, generating a perspective transformation matrix according to the third position information and the fourth position information.

[0084] Among them, the second standard image can be an image captured by the image capture device from the first perspective of the container sample.

[0085] Specifically, a trained target detection model is deployed in the terminal. The first standard image is input into the target detection model to detect the container top contour of the container sample in the first standard image, and the third position information of the container top contour in the first standard image is obtained. The second standard image is input into the target detection model to detect the container top contour of the container sample in the second standard image, and the fourth position information of the container top contour in the second standard image is obtained. Arithmetic processing is performed on the third position information of the container top contour in the second perspective and the fourth position information of the container top contour in the first perspective to generate a perspective transformation matrix.

[0086] In this embodiment, by obtaining the top contour of the container sample from the second perspective and the top contour of the container sample from the first perspective, and generating a perspective transformation matrix based on the position information of the top contour of the container from different perspectives, it is possible to subsequently use the perspective transformation matrix to convert the image of the container to be detected from the second perspective into the target container image from the first perspective, thereby improving the detection accuracy of the object height.

[0087] In an embodiment disclosed in the present application, step S106, performing perspective transformation processing on the image of the container to be detected according to the first position information and the preset perspective transformation matrix to obtain the target container image, includes:

[0088] Step S302, performing perspective transformation processing on the image of the container to be detected according to the first position information and the perspective transformation matrix to obtain the image of the container to be detected after perspective transformation processing.

[0089] Step S304, determining the image size of the second standard image according to the fourth position information.

[0090] Step S306, performing scaling processing on the image of the container to be detected after perspective transformation processing according to the image size to obtain the target container image.

[0091] Among them, the image size can be used to represent the pixel aspect ratio of the second standard image.

[0092] Specifically, a preset perspective transformation matrix is stored in the terminal, and the internal area of the container in the image of the container to be detected that needs to be subjected to perspective transformation is determined according to the first position information. Perspective transformation processing is performed on each pixel point in the internal area of the container using the perspective transformation matrix to obtain the image of the container to be detected after perspective transformation. Arithmetic processing is performed on the fourth position information of the pixel points at the four corners of the top contour of the container in the second standard image to determine the image size of the second standard image. The image of the container to be detected after perspective transformation processing is scaled according to the image size of the second standard image until the image size of the scaled image is consistent with the image size of the second standard image to obtain the target container image.

[0093] In this embodiment, by determining the image size of the second standard image and performing scaling processing on the image of the container to be detected after perspective transformation processing according to the image size to obtain the target container image, it is possible to facilitate subsequent determination of the height of the target object using the target container image, thereby improving the detection efficiency of the object height.

[0094] In an embodiment disclosed in the present application, the container sample in the second standard image is marked with a plurality of reference scales in the horizontal direction. Step S110 of generating the height of the target object according to the comparison result between the target position information and the plurality of reference scales includes: generating the height of the target object according to the comparison result between the target position information and the plurality of reference scales in the second standard image.

[0095] Specifically, the terminal may compare the target position information in the target container image with the reference positions corresponding to the plurality of reference scales in the horizontal direction in the second standard image in the vertical direction, generate the comparison results between the target position information and each reference scale, determine the reference scale with the highest matching degree with the target position information, and use the object height corresponding to the reference scale with the highest matching degree as the height of the target object. Alternatively, the terminal may also perform arithmetic processing on the target position information of the edge of the target object in the target container image to determine the edge pixel length corresponding to the target object. Compare the edge pixel length with the reference pixel lengths corresponding to the plurality of reference scales in the second standard image, determine the reference scale with the highest matching degree with the edge pixel length of the target object, and use the object height corresponding to the reference scale with the highest matching degree as the height of the target object.

[0096] In this embodiment, by comparing the target position information of the target object with the plurality of reference scales marked on the container sample, and generating the height of the target object according to the comparison result between the target position information in the target container image and the plurality of reference scales in the second standard image, it is possible to quickly generate the height of the target object based on the comparison result between the target position information and the reference scale, improving the detection efficiency of the object height.

[0097] In an embodiment disclosed in the present application, step S104 of determining the first position information of the top contour of the container and the second position information of the edge of the target object in the container to be detected includes: inputting the container image to be detected into the detection model to obtain the first position information.

[0098] Among them, the detection model can be obtained by the terminal training an untrained target detection model according to the sample container image and the target label corresponding to the sample container image.

[0099] The target label can be obtained by the terminal performing Gaussian blur processing on the original label of the sample container image.

[0100] The original label can be obtained by the terminal marking a plurality of boundary points at the top contour of the sample container in the sample container image in a preset order.

[0101] Specifically, a trained detection model is deployed in the terminal. The container image to be detected is input into the detection model, and the detection model sequentially detects multiple boundary points at the top contour of the container in the container image to be detected, obtains the pixel point position information corresponding to the multiple boundary points at the top contour of the container in the container image to be detected, and uses the obtained multiple pixel point position information as the first position information of the top contour of the container.

[0102] In one example, the training method of the detection model is described as follows: The terminal obtains multiple sample container images, and sequentially labels the upper left boundary point, upper right boundary point, lower right boundary point, and lower left boundary point at the top contour of the sample container in each sample container image in clockwise order to generate the original label of each sample container image. The following processing is performed on each sample container image: The multiple original labels in the sample container image are scaled, and the scaled original labels are subjected to Gaussian blur processing to obtain the target label corresponding to the sample container image. The multiple sample container images are grouped to determine the sample container images in the sample training set and the sample container images in the sample test set. The multiple sample container images in the sample training set and the target label corresponding to each sample container image in the sample training set are used as the input data of the detection model, and the untrained detection model is iteratively trained, and the parameters of the detection model are optimized using the stochastic gradient descent algorithm. The multiple sample container images in the sample test set and the target label corresponding to each sample container image in the sample test set are used as the test data of the detection model, and the detection model obtained after iterative training is tested, and the loss function of the detection model is determined using the mean square variance calculation method. The above iterative training process is repeated until the loss function of the detection model satisfies the iterative stop condition, or until the number of iterations satisfies the iterative stop condition, to obtain the trained detection model.

[0103] In this embodiment, by using the sample container image and the target label corresponding to the sample container image to train the untrained target detection model to obtain the trained detection model, and inputting the container image to be detected into the trained detection model to determine the first position information of multiple boundary points at the top contour of the container, the accuracy of the first position information can be improved, thereby improving the accuracy of the subsequent perspective transformation processing of the container image to be detected using the first position information to generate the target container image.

[0104] In addition, in the existing sample container image annotation methods, since the boundary points of the top contour of the sample container are difficult to be accurately defined by a certain pixel position, there is a problem of great difficulty in annotating the boundary points. In this embodiment, the original label of the sample container image is subjected to Gaussian blur processing to generate the target label of the sample container image, and the target label and the sample container image obtained after Gaussian blur processing are used to train the untrained detection model, which can reduce the interference in the training process of the detection model, make the network convergence of the detection model better, and thus improve the detection accuracy of the detection model.

[0105] In another embodiment disclosed in the present application, step S104, determining the first position information of the container top contour and the second position information of the edge of the target object in the container to be detected image, includes:

[0106] Step S402, performing image segmentation processing on the container to be detected image to obtain the image inside the container.

[0107] Step S404, performing straight line detection processing on the image inside the container to generate the straight line detection result of the image inside the container.

[0108] Step S406, performing clustering processing on the straight line detection result to obtain the line segment clustering result, and determining the area where the target object is located according to the line segment clustering result.

[0109] Step S408, performing edge detection on the area where the target object is located to determine the second position information.

[0110] Specifically, the terminal can perform image segmentation processing on the area inside the container and the background area outside the area inside the container in the container to be detected image to obtain the image inside the container of the area inside the container. An edge detection operator is used to perform edge detection processing on the image inside the container to obtain the edge detection result of the image inside the container. A straight line detection algorithm is used to perform straight line detection on the straight line segments in the edge detection result to generate the straight line detection result of the image inside the container. A line segment filtering threshold is used to filter the straight line detection result to obtain the filtered straight line detection result. A density clustering algorithm is used to perform clustering processing on the filtered straight line detection result to obtain the line segment clustering result of the image inside the container, and the clustering result with the largest number of straight line segments in the line segment clustering result is used as the category to which the target object belongs to obtain the area where the target object is located. Multiple edge detection algorithms are respectively used to perform edge detection on the area where the target object is located to obtain the edge detection result of the area where the target object is located corresponding to each edge detection algorithm. According to the edge detection result corresponding to each edge detection algorithm, the second position information of the edge of the target object is determined.

[0111] In one example, the terminal can generate a line segment length distribution diagram of the line detection result according to the line segment length of each line segment in the line detection result, and determine the line segment length with the most occurrences and the line segment length with the fewest occurrences in the line segment length distribution diagram. Determine the line segment filtering threshold according to the line segment length with the most occurrences and the line segment length with the fewest occurrences. The line segment filtering threshold can be used to filter out the line segments with relatively low occurrences in the line detection result.

[0112] In one example, the terminal can perform edge detection on the region where the target object is located by using the convex hull detection algorithm, the minimum bounding rectangle detection algorithm, and the maximum inscribed rectangle detection algorithm respectively, to obtain the convex hull detection result corresponding to the convex hull detection algorithm, the minimum bounding rectangle detection result corresponding to the minimum bounding rectangle detection algorithm, and the maximum inscribed rectangle detection result corresponding to the maximum inscribed rectangle detection algorithm. Determine the midpoint position of the region between the minimum bounding rectangle detection result and the maximum inscribed rectangle detection result. Use the convex hull detection result as the predicted edge of the region where the target object is located. Obtain the pixel position information at the intersection of the horizontal line where the midpoint position is located and the predicted edge, and use the pixel position information at the intersection as the second position information of the upper edge of the target object.

[0113] In this embodiment, by performing line detection processing on the image inside the container of the image to be detected, determining the region where the target object is located according to the line detection result of the image inside the container, performing edge detection on the region where the target object is located, and determining the second position information, it is possible to detect the second position information of the edge of the target object based on the line features in the image to be detected, which has better robustness and higher detection accuracy.

[0114] In an embodiment disclosed in the present application, a method for detecting the height of an object is provided, including:

[0115] Step S502, in response to an image acquisition operation, display an image acquisition page, display an image mask on the image acquisition page, and obtain an image of a container to be detected that matches the image mask through the image acquisition page.

[0116] Specifically, an image mask is pre-stored in the terminal. In response to an image acquisition operation triggered by the user, display an image acquisition page, and display an image mask on the image acquisition page. Obtain a container image through the image acquisition page, and determine an image of a container to be detected that matches the image mask according to the similarity between the container image and the image mask. The specific operation for obtaining the image of the container to be detected can be implemented with reference to the method for obtaining the container to be detected provided in the above embodiment, and will not be specifically elaborated here. In one example, as Figure 5a shown, a schematic diagram of the steps for generating an image mask is provided. Figure 5aOn the left is a first standard image obtained by collecting an image of an open trash can sample from a second perspective, where the open trash can sample has volume graduations. Figure 5a On the right is an image mask generated based on the first standard image on the left. In another example, as Figure 5b shown, a schematic diagram of the steps for obtaining an image of a container to be detected is provided. Figure 5b An image mask, an image acquisition prompt message "Please take a photo of the trash can", and an image acquisition button are displayed on the image acquisition page on the left. Figure 5b The image of the container to be detected on the right is Figure 5b an image obtained from the image acquisition page on the left and matching the image mask.

[0117] Step S504: Input the image of the container to be detected into the detection model to obtain first position information of the top contour of the container in the image of the container to be detected.

[0118] Specifically, a trained detection model is deployed in the terminal. The image of the container to be detected is input into the detection model, and the detection model sequentially detects the upper left boundary point, upper right boundary point, lower right boundary point, and lower left boundary point at the top contour of the container in the image of the container to be detected, and obtains the pixel point position information corresponding to the upper left boundary point, upper right boundary point, lower right boundary point, and lower left boundary point at the top contour of the container in the image of the container to be detected. The obtained multiple pixel point position information is used as the first position information of the top contour of the container. The training operation of the specific detection model can be implemented with reference to the training method of the detection model provided in the above embodiments, and will not be specifically elaborated here. In one example, as Figure 5c shown, a schematic diagram of the steps for determining the first position information is provided.

[0119] Step S506: Perform a straight line detection process on the image of the object inside the container in the image of the container to be detected, and determine second position information of the edge of the target object inside the container according to the straight line detection result of the image of the object inside the container.

[0120] Specifically, the terminal can perform image segmentation processing on the container image to be detected to obtain the inner image of the container area. The Canny operator is used to perform edge detection processing on the inner image of the container to obtain the edge detection result of the inner image of the container. The FLD algorithm (Fast Line Detection) is used to perform line detection on the line segments in the edge detection result to generate the line detection result of the inner image of the container. The line segment filtering threshold is determined according to the occurrence frequency of the line segment lengths in the line detection result, and the line detection result is filtered using the line segment filtering threshold. The DBSCAN algorithm (Density-Based Spatial Clustering of Applications with Noise) is used to perform clustering processing on the filtered line detection result to obtain the line segment clustering result of the inner image of the container. The area where the target object is located is determined according to the line segment clustering result, and edge detection is performed on the area where the target object is located to determine the second position information of the upper edge of the target object. The specific determination operation of the second position information can be implemented with reference to the determination method of the second position information provided in the above embodiments, and will not be specifically elaborated here. In one example, as Figure 5d shown, a schematic diagram of the second position information determination step is provided. The terminal performs convex hull detection on the area where the target object is located to obtain the edge of the target object area. The minimum bounding rectangle detection algorithm and the maximum inscribed rectangle detection algorithm are respectively used to detect the area where the target object is located to determine the minimum bounding rectangle area and the maximum inscribed rectangle area corresponding to the area where the target object is located. The midpoint of the area between the minimum bounding rectangle area and the maximum inscribed rectangle area is determined, the intersection point of the horizontal line where the midpoint is located and the edge of the target object area obtained by convex hull detection is determined, the upper edge of the area where the target object is located is determined according to the intersection point, and the second position information of the upper edge of the target object is obtained according to the position information of the intersection point.

[0121] In another example, the terminal can use any one of various edge detection algorithms, such as the HED algorithm (Holistically-nested Edge Detection) and the PidiNet detection algorithm of pixel difference network, to perform edge detection processing on the image in the container to obtain an edge detection result. Any one of various line detection algorithms, such as the LSD algorithm (LineSegment Detector) and the LSM algorithm (a line detection algorithm), can be used to perform line detection on the line segments in the edge detection result to obtain a line detection result. Any one of various density clustering algorithms, such as the Optics algorithm (Ordering points to identify the clustering structure) and the MDCA algorithm (Maximum Density Clustering Application), can be used to perform clustering processing on the filtered line detection result to obtain a line segment clustering result.

[0122] Step S508: Perform perspective transformation processing on the container image to be detected according to the first position information and the perspective transformation matrix, and perform scaling processing on the container image to be detected after perspective transformation processing according to the image size of the second standard image to obtain a target container image.

[0123] Specifically, a preset perspective transformation matrix is stored in the terminal. Determine the internal area of the container in the container image to be detected that needs to be subjected to perspective transformation according to the first position information. Perform perspective transformation processing on each pixel point in the internal area of the container using the perspective transformation matrix to obtain the container image to be detected after perspective transformation. Perform arithmetic processing on the fourth position information of the container top contour on the second standard image to determine the image size of the second standard image. Perform scaling processing on the container image to be detected after perspective transformation processing according to the image size of the second standard image to obtain a target container image. The specific generation operation of the target container image can be implemented with reference to the target container image generation method provided in the above embodiment, and will not be specifically elaborated here. In one example, as Figure 5e shown, a schematic diagram of the target container image generation steps is provided.

[0124] In another example, as Figure 5f shown, a schematic diagram of the perspective transformation matrix generation steps is provided. The terminal can generate a perspective transformation matrix according to the following formula:

[0125]

[0126]

[0127]

[0128]

[0129] T 3 = [a 31 a 32

[0130] wherein, X ABCD is the abscissa of the pixel points A, B, C, and D at the four corners of the top contour of the container on the second standard image, Y ABCD is the ordinate of the pixel points A, B, C, and D at the four corners of the top contour of the container on the second standard image, x abcd is the abscissa of the pixel points a, b, c, and d at the four corners of the top contour of the container on the first standard image, y abcd is the ordinate of the pixel points a, b, c, and d at the four corners of the top contour of the container on the first standard image, Transform is the perspective transformation matrix, T 1 is the matrix for image linear transformation, T 2 is the matrix for generating the perspective transformation of the image, T 3 is the matrix for image translation.

[0131] Step S510, determine the target position information corresponding to the second position information in the target container image, and generate the height of the target object according to the comparison result between the target position information and multiple reference scales.

[0132] Specifically, the terminal uses the perspective transformation matrix to perform arithmetic processing on the second position information of the edge of the target object, and generates the target position information corresponding to the second position information in the target container image. Compare the target position information with multiple reference scales in the horizontal direction in the second standard image, determine the reference scale with the highest matching degree with the target position information, and use the height corresponding to the reference scale with the highest matching degree as the height of the target object. The specific generation operation of the height of the target object can be implemented with reference to the method for generating the height of the target object provided in the above embodiments, and will not be specifically elaborated here. In one example, as Figure 5g shown, a schematic diagram of the steps for generating the height of the target object is provided. Compare the upper edge of the target object in the target container image with the reference scale in the second standard image, and generate the height of the target object according to the comparison result.

[0133] ​In this embodiment, by obtaining the image of the container to be detected through an image mask, the image acquisition angle of the image of the container to be detected can be unified, thereby improving the processing efficiency of the subsequent image of the container to be detected; by using a detection model to determine the first position information of the top contour of the container in the image of the container to be detected, the accuracy of the first position information can be improved; by performing a straight-line detection process on the image inside the container of the image of the container to be detected, determining the area where the target object is located according to the straight-line detection result of the image inside the container, and performing edge detection on the area where the target object is located to determine the second position information, the second position information of the edge of the target object can be detected based on the line features in the image of the container to be detected, which has better robustness and higher detection accuracy; by using a perspective transformation matrix to convert the image of the container to be detected under the second perspective into the target container image under the first perspective, and generating the height of the target object according to the comparison result between the target position information on the target container image and multiple reference scales in the second standard image under the first perspective, the height of the target object can be quickly generated based on the comparison result between the target position information and the reference scales in the images under the same perspective, which not only improves the detection efficiency of the object height, but also improves the detection accuracy of the object height at the same time.

[0134] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps does not have a strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0135] Based on the same inventive concept, an embodiment of the present application also provides a detection device for the height of an object for implementing the above-mentioned detection method for the height of an object. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the detection device for the height of an object provided below can refer to the limitations on the detection method for the height of an object in the above text, and will not be repeated here.

[0136] In an embodiment disclosed in the present application, as Figure 6 shown, a detection device 600 for the height of an object is provided, including: an image acquisition module 602, a first determination module 604, a perspective transformation module 606, a second determination module 608, and a height generation module 610, where:

[0137] An image acquisition module 602, configured to acquire an image of a container to be detected.

[0138] A first determination module 604, configured to determine first position information of the top contour of the container in the image of the container to be detected, and second position information of the edge of the target object in the container.

[0139] A perspective transformation module 606, configured to perform perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix, to obtain a target container image, where the perspective transformation matrix is used to convert the image of the container to be detected into an image under a first perspective.

[0140] A second determination module 608, configured to determine target position information corresponding to the second position information in the target container image.

[0141] A height generation module 610, configured to generate the height of the target object according to the comparison result between the target position information and multiple reference scales.

[0142] In an embodiment disclosed in the present application, the image acquisition module 602 includes: a mask display unit, configured to display an image acquisition page in response to an image acquisition operation, and display an image mask in the image acquisition page, where the image mask is generated according to a first standard image, and the first standard image is obtained by photographing a container sample from a second perspective; a mask matching unit, configured to acquire the image of the container to be detected that matches the image mask through the image acquisition page.

[0143] In an embodiment disclosed in the present application, the perspective transformation module 606 includes: a perspective transformation matrix generation unit, configured to detect the first standard image to obtain third position information of the top contour of the container in the first standard image; detect a second standard image to obtain fourth position information of the top contour of the container in the second standard image, where the second standard image is obtained by photographing a container sample from a first perspective; generate a perspective transformation matrix according to the third position information and the fourth position information.

[0144] In an embodiment disclosed in the present application, the perspective transformation module 606 further includes: a size scaling unit, configured to perform perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix, to obtain the image of the container to be detected after perspective transformation processing; determine the image size of the second standard image according to the fourth position information; perform scaling processing on the image of the container to be detected after perspective transformation processing according to the image size, to obtain a target container image.

[0145] In an embodiment disclosed in the present application, the container sample in the second standard image is marked with a plurality of reference scales in the horizontal direction. The height generation module 610 is further configured to: generate the height of the target object according to the target position information and the comparison result between the plurality of reference scales in the second standard image.

[0146] In an embodiment disclosed in the present application, the first determination module 604 includes: a contour detection unit, configured to input the container image to be detected into a detection model to obtain first position information. The detection model is trained according to the sample container image and the target label corresponding to the sample container image. The target label is obtained by performing Gaussian blur processing on the original label of the sample container image. The original label is obtained by labeling a plurality of boundary points of the container top contour in the sample container image in a preset order.

[0147] In an embodiment disclosed in the present application, the first determination module 604 includes: an edge detection unit, configured to perform image segmentation processing on the container image to be detected to obtain an inner-container image; perform straight-line detection processing on the inner-container image to generate a straight-line detection result of the inner-container image; perform clustering processing on the straight-line detection result to obtain a line segment clustering result, determine the area where the target object is located according to the line segment clustering result; perform edge detection on the area where the target object is located to determine second position information.

[0148] Each module in the above-described apparatus for detecting the height of an object can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor in the computer device in the form of hardware or be independent of it, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0149] In an embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 7As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for detecting the height of an object. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0150] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0151] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0152] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0153] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0154] It should be noted that the data involved in this application (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data that have been fully authorized by all parties.

[0155] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0156] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0157] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for detecting the height of an object, characterized in that, the method includes: obtaining an image of the container to be detected; determining first position information of the top contour of the container in the image of the container to be detected, and second position information of the edge of the target object in the container; performing perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix, to obtain a target container image, where the perspective transformation matrix is used to convert the image of the container to be detected into an image under a first viewing angle; determining target position information corresponding to the second position information in the target container image; generating the height of the target object according to a comparison result between the target position information and a plurality of reference scales, where the reference scales are reference scales of the container; wherein, the obtaining of the image of the container to be detected includes: in response to an image acquisition operation, displaying an image acquisition page, and displaying an image mask on the image acquisition page, where the image mask is generated according to a first standard image, and the first standard image is obtained by photographing a container sample from a second viewing angle; obtaining the image of the container to be detected that matches the image mask through the image acquisition page.

2. The method according to claim 1, characterized in that, the generation method of the perspective transformation matrix includes: detecting the first standard image to obtain third position information of the top contour of the container in the first standard image; detecting a second standard image to obtain fourth position information of the top contour of the container in the second standard image, where the second standard image is obtained by photographing the container sample from the first viewing angle; generating the perspective transformation matrix according to the third position information and the fourth position information.

3. The method according to claim 2, characterized in that, the performing perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix to obtain a target container image includes: performing perspective transformation processing on the image of the container to be detected according to the first position information and the preset perspective transformation matrix to obtain the image of the container to be detected after perspective transformation processing; determining the image size of the second standard image according to the fourth position information; performing scaling processing on the image of the container to be detected after perspective transformation processing according to the image size to obtain the target container image.

4. The method according to any one of claims 1 to 3, characterized in that, the determining of the first position information of the top contour of the container in the image of the container to be detected includes: inputting the image of the container to be detected into a detection model to obtain the first position information, where the detection model is trained according to a sample container image and a target label corresponding to the sample container image, and the target label is obtained by performing Gaussian blur processing on an original label of the sample container image, and the original label is obtained by labeling a plurality of boundary points of the top contour of the container in the sample container image in a preset order.

5. The method according to any one of claims 1 to 3, characterized in that, Determine the second position information of the edge of the target object in the container, including: Perform image segmentation processing on the image of the container to be detected to obtain the image inside the container; Perform line detection processing on the image inside the container to generate the line detection result of the image inside the container; Perform clustering processing on the line detection result to obtain the line segment clustering result, and determine the area where the target object is located according to the line segment clustering result; Perform edge detection on the area where the target object is located to determine the second position information.

6. A device for detecting the height of an object, characterized in that, the device includes: An image acquisition module for acquiring an image of a container to be detected; A first determination module for determining the first position information of the top contour of the container in the image of the container to be detected and the second position information of the edge of the target object in the container; A perspective transformation module for performing perspective transformation processing on the image of the container to be detected according to the first position information and a preset perspective transformation matrix to obtain a target container image, and the perspective transformation matrix is used to convert the image of the container to be detected into an image under the first perspective; A second determination module for determining the target position information corresponding to the second position information in the target container image; A height generation module for generating the height of the target object according to the comparison result between the target position information and a plurality of reference scales, and the reference scale is the reference scale of the container; Wherein, the image acquisition module includes: A mask display unit for responding to an image acquisition operation, displaying an image acquisition page, and displaying an image mask in the image acquisition page, and the image mask is generated according to a first standard image, and the first standard image is obtained by photographing a container sample from a second perspective; A mask matching unit for obtaining the image of the container to be detected that matches the image mask through the image acquisition page.

7. The device according to claim 6, characterized in that, the perspective transformation module includes: A perspective transformation matrix generation unit for detecting the first standard image to obtain the third position information of the top contour of the container in the first standard image; detecting a second standard image to obtain the fourth position information of the top contour of the container in the second standard image, and the second standard image is obtained by photographing the container sample from the first perspective; generating the perspective transformation matrix according to the third position information and the fourth position information.

8. A computer device includes a memory and a processor, and the memory stores a computer program, characterized in that, when the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product includes a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images