Method, device, equipment and storage medium for detecting the position of a vat for discharging mash

Abstract

The present application provides a method, device, equipment, and storage medium for detecting the position of a vat to be discharged from mash, specifically relating to the technical field of detecting the position of a vat to be discharged from mash. The method comprises: a processor inputting the coordinates of the i-th data point of a target point cloud image into a residual model to obtain a residual value of the i-th data point of the target point cloud image; based on the Huber loss function, calculating the loss value of the i-th data point of the target point cloud image according to the residual value of the i-th data point of the target point cloud image; summing I data points of the target point cloud image to obtain a total loss value of I data points of the target point cloud image; calculating a target gradient according to the total loss value of I data points of the target point cloud image; iterating the initial parameters of the target gradient to obtain target parameters; and determining the position of the vat to be discharged from mash according to N target parameters. This method can avoid the accuracy of the position determination of the vat to be discharged from mash being affected by excessive outlier data.

Description

Technical Field

[0001] The present application relates to the technical field of position detection of a vat for discharging mash, and in particular to a method, device, equipment and storage medium for position detection of a vat for discharging mash. Background Art

[0002] Currently, in the field of liquor brewing, there are two main types of liquor fermentation vessels: cellars and ground vats. Cellars are mainly used for the fermentation of strong-flavor liquor and sauce-flavor liquor. After fermentation is complete, it is more convenient to automate the discharge of the mash. Ground vats are mostly used for the fermentation of light-flavor liquor and Laobaigan-flavor liquor. Because they are buried underground and have a small internal space, the discharge environment is relatively complex. To achieve the automation of the discharge of mash from the ground vat, not only a reasonable and safe discharge terminal is required, but also the precise sensing of the position of the ground vat. Moreover, there are many ground vats at the discharge site, making it difficult to accurately sense the position of the ground vat to be discharged. Therefore, an efficient and accurate method of sensing the spatial position of the ground vat can greatly improve the working efficiency of the mash discharge equipment and facilitate the automation of the liquor production process.

[0003] In the prior art, the processor uses the least squares method to fit the center point coordinates in the three-dimensional point cloud image of the vat to be used for mash production, and determines the position of the vat to be used for mash production based on the center point coordinates in the three-dimensional point cloud image of the vat to be used for mash production.

[0004] However, the three-dimensional point cloud image of the earthen jar for mash to be discharged in the above method includes many outliers, which have a great influence on the accuracy of the center point coordinates in the obtained three-dimensional point cloud image of the earthen jar for mash to be discharged. The accurate position of the earthen jar for mash to be discharged cannot be obtained by the least squares method, which is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of the present application is to provide a method, device, equipment and storage medium for detecting the position of a vat for discharging mash to solve the above problems and achieve accurate positioning of the vat for discharging mash.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting the position of a vat for discharging mash, which is applied to a processor, and the method includes:

[0007] The processor obtains a target point cloud image; wherein the target point cloud image is a three-dimensional point cloud image of a cross section at an n-th height of the jar where the mash is to be discharged;

[0008] The processor inputs the coordinates of the i-th data point of the target point cloud image into the residual model to obtain the residual value of the i-th data point of the target point cloud image;

[0009] The processor calculates the loss value of the i-th data point of the target point cloud image based on the Huber loss function and the residual value of the i-th data point of the target point cloud image:

[0010]

[0011] in, is the loss value of the i-th data point of the target point cloud image, is the residual value of the i-th data point of the target point cloud image, A threshold to control the smoothness of the loss function;

[0012] The processor sums I data points of the target point cloud image to obtain a total loss value of the I data points of the target point cloud image;

[0013] The processor calculates a target gradient based on the total loss value of I data points of the target point cloud image; wherein the target gradient is the gradient of the total loss value of I data points of the target point cloud image;

[0014] The processor iterates the initial parameters of the target gradient to obtain target parameters; wherein the target parameters include the coordinates of the center of the ellipse model corresponding to the target point cloud image;

[0015] The processor determines the position of the vat where the mash is to be discharged according to N target parameters.

[0016] Preferably, the step of the processor acquiring the target point cloud image includes:

[0017] The processor acquires a depth image of each ground cylinder and a color image of each ground cylinder;

[0018] The processor labels the color image of each ground cylinder to obtain a target color image of each ground cylinder;

[0019] The processor inputs the target color image of each ground vat into the trained neural network model, so that the neural network model outputs the mask image of the ground vat to be used for mash production;

[0020] The processor performs AND operations on pixel values ​​of the mask image of the ground vat to be discharged and pixel values ​​of the depth images of J ground vats in sequence to obtain J first operation results;

[0021] If the j-th first operation result is 1, the processor determines the depth image of the j-th ground vat as the depth image of the ground vat to be used for discharging mash;

[0022] The processor performs an AND operation on the pixel values ​​of the mask image of the ground vat to be discharged and the pixel values ​​of the color images of the M ground vats in sequence to obtain M second operation results;

[0023] If the mth second operation result is 1, the processor determines the color image of the mth ground vat as the color image of the ground vat to be used for discharging mash;

[0024] The processor transforms the coordinates of the pixel points of the depth image of the vat to be discharged, to obtain three-dimensional space points corresponding to the pixel points of the depth image of the vat to be discharged;

[0025] The processor fuses the three-dimensional space points corresponding to the pixel points of the depth image of the jar to be discharged from the mash with the pixel points of the color image of the jar to be discharged from the mash to obtain a three-dimensional point cloud image of the jar to be discharged from the mash;

[0026] The processor cuts the three-dimensional point cloud image of the vat to be used for discharging mash along a horizontal direction to obtain the target point cloud image.

[0027] Preferably, the method further comprises:

[0028] If the j-th first operation result is 0, the processor determines the depth image of the j-th ground vat as a depth image of a ground vat not to be used for discharging mash.

[0029] Preferably, the method further comprises:

[0030] If the mth second operation result is 0, the processor determines the color image of the mth ground vat as a color image of a ground vat that is not to be used for discharging mash.

[0031] Preferably, the processor determines the position of the vat for discharging the mash according to N target parameters, comprising:

[0032] The processor calculates the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images and the average value of the vertical coordinates of the center of the ellipse model corresponding to the N target point cloud images according to the following formula:

[0033]

[0034] in, is the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images, is the average value of the ordinate of the center of the ellipse model corresponding to the N target point cloud images, is the horizontal coordinate of the center of the ellipse model corresponding to the nth target point cloud image, is the ordinate of the center of the ellipse model corresponding to the nth target point cloud image;

[0035] The processor determines the coordinates of the center of the ellipse model corresponding to the N target point cloud images according to the average value of the abscissa of the center of the ellipse model corresponding to the N target point cloud images and the average value of the ordinate of the center of the ellipse model corresponding to the N target point cloud images;

[0036] The processor determines the coordinates of the center of the ellipse model corresponding to the N target point cloud images as the position of the vat where the mash is to be discharged.

[0037] Preferably, the method further comprises:

[0038] The processor obtains an initial depth image of each ground cylinder sent by the TOF camera;

[0039] The processor filters the initial depth image of each ground cylinder through a Gaussian filter to obtain a depth image of each ground cylinder.

[0040] The method for detecting the position of the jar for discharging mash provided in this application brings the following beneficial effects:

[0041] The present application provides a method for detecting the position of a vat to be discharged from a mash. In this method, a processor is based on the Huber loss function. When the experimental data is relatively close to outliers, the processor uses the absolute error loss calculation method to obtain the loss value of the i-th data point of the target point cloud image according to the residual value of the i-th data point of the target point cloud image. When the experimental data is relatively close to outliers, the processor uses the square error loss calculation method to obtain the loss value of the i-th data point of the target point cloud image according to the residual value of the i-th data point of the target point cloud image. This method has high accuracy and can avoid the accuracy of determining the position of the vat to be discharged from a mash due to excessive outlier data, so that when the mash is discharged, the automatic handling equipment can accurately identify the position of the vat to be discharged from the mash and remove the mash from the vat.

[0042] In a second aspect, the present application further provides a device for detecting the position of a vat for discharging mash, the device comprising:

[0043] An acquisition module is used to acquire a target point cloud image; wherein the target point cloud image is a three-dimensional point cloud image of a cross section at the nth height of the base vat where the mash is to be discharged;

[0044] a data processing module, configured to input the coordinates of the i-th data point of the target point cloud image into a residual model to obtain a residual value of the i-th data point of the target point cloud image;

[0045] A calculation module is used to calculate the loss value of the i-th data point of the target point cloud image based on the Huber loss function according to the residual value of the i-th data point of the target point cloud image:

[0046]

[0047] in, is the loss value of the i-th data point of the target point cloud image, is the residual value of the i-th data point of the target point cloud image, A threshold to control the smoothness of the loss function;

[0048] The calculation module is further configured to sum I data points of the target point cloud image to obtain a total loss value of the I data points of the target point cloud image; and calculate a target gradient based on the total loss value of the I data points of the target point cloud image; wherein the target gradient is the gradient of the total loss value of the I data points of the target point cloud image;

[0049] The data processing module is further configured to iterate the initial parameters of the target gradient to obtain target parameters; wherein the target parameters include the coordinates of the center of the ellipse model corresponding to the target point cloud image;

[0050] The detection module is used to determine the position of the vat for discharging the mash according to N target parameters.

[0051] The position detection device for the ground vat for discharging mash provided in the embodiment of the present application has the same technical features as the position detection method for the ground vat for discharging mash provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0052] In a third aspect, the present application provides a computing device, including a memory and a processor;

[0053] One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as described in any one of the first aspects.

[0054] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program for executing the method as described in any one of the first aspects.

[0055] In a fifth aspect, the present application provides a computer program product, which includes one or more computer instructions. When the computer instructions are executed by a computer, the computer executes the method as described in any one of the first aspects.

[0056] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application are realized and obtained by the structures particularly pointed out in the description and drawings.

[0057] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0059] Figure 1 A schematic flow chart of a method for detecting the position of a vat for discharging mash provided in an embodiment of the present application;

[0060] Figure 2 A flowchart of a method for a processor to obtain a target point cloud image provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a target color image of each ground cylinder provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of an AND operation provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of another AND operation provided in an embodiment of the present application;

[0064] Figure 6 A schematic structural diagram of a device for detecting the position of a vat for discharging mash provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of the computing device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0067] To facilitate understanding of this embodiment, the embodiments of this application are described in detail below.

[0068] The embodiment of the present application provides a method for detecting the position of a vat to be used for discharging mash, which is applied to a processor, such as Figure 1 As shown, Figure 1 A flow chart of a method for detecting the position of a jar for discharging mash provided in an embodiment of the present application. The method comprises the following steps:

[0069] S101: The processor obtains a target point cloud image.

[0070] The target point cloud image is a three-dimensional point cloud image of the cross section at the nth height of the jar where the mash is to be discharged.

[0071] Specifically, the above-mentioned target point cloud image is a collection of a large number of discrete points in space, including the three-dimensional coordinate (X, Y, Z) information of the cross-sections of the ground vat at different heights to be used for the mash. It directly reflects the true shape and structure of the cross-section of the ground vat to be used for the mash, and provides technicians with detailed geometric, shape and scale information of the ground vat to be used for the mash.

[0072] S102 , the processor inputs the coordinates of the i-th data point of the target point cloud image into the residual model to obtain the residual value of the i-th data point of the target point cloud image.

[0073] Specifically, the calculation formula (residual model) of the residual value of the i-th data point of the target point cloud image is as follows:

[0074]

[0075] in, is the residual value of the i-th data point of the target point cloud image, is the length of the major axis of the cross section at the nth height of the vat where the mash is to be discharged, is the length of the minor semi-axis of the cross section at the nth height of the jar where the mash is to be discharged, is the horizontal coordinate of the i-th data point of the target point cloud image, is the ordinate of the i-th data point of the target point cloud image, is the horizontal coordinate of the center point of the cross section at the nth height of the vat where the mash is to be discharged, It is the ordinate of the center point of the cross section at the nth height of the vat where the mash is to be discharged.

[0076] The following example illustrates the calculation process of the residual value of the i-th data point of the target point cloud image. For example, the horizontal coordinate of the i-th data point of the target point cloud image is 5, the vertical coordinate of the i-th data point of the target point cloud image is 8, the major semi-axis length of the cross section at the n-th height of the earthen jar to be discharged is 2, the minor semi-axis length of the cross section at the n-th height of the earthen jar to be discharged is 1.5, the horizontal coordinate of the center point of the cross section at the n-th height of the earthen jar to be discharged is 6, and the vertical coordinate of the center point of the cross section at the n-th height of the earthen jar to be discharged is 9. Then the residual value of the i-th data point of the target point cloud image is -0.31.

[0077] S103 , the processor calculates a loss value of the i-th data point of the target point cloud image based on the Huber loss function and the residual value of the i-th data point of the target point cloud image.

[0078]

[0079] in, is the loss value of the i-th data point of the target point cloud image, is the residual value of the i-th data point of the target point cloud image, is the threshold for controlling the smoothness of the loss function.

[0080] Specifically, the following example introduces the calculation process of the loss value of the i-th data point of the target point cloud image. For example, the threshold for controlling the smoothness of the loss function is set to 1. If the residual value of the i-th data point of the target point cloud image is -0.31, -0.31 , then substitute -0.31 into Calculate the loss value of the i-th data point of the target point cloud image and get 0.04805; if the residual value of the i-th data point of the target point cloud image is 3.25, 3.25 , then substitute 3.25 into Calculation is performed in , and the loss value of the i-th data point of the target point cloud image is obtained to be 2.75.

[0081] With such settings, when the experimental data are mostly near outliers, that is, when the residual value of the i-th data point of the target point cloud image is greater than or equal to the threshold value that controls the smoothness of the loss function, the absolute error loss calculation method is used to obtain the loss value of the i-th data point of the target point cloud image; when the experimental data are less near outliers, that is, when the residual value of the i-th data point of the target point cloud image is less than the threshold value that controls the smoothness of the loss function, the square error loss calculation method is used to obtain the loss value of the i-th data point of the target point cloud image.

[0082] The Huber loss function combines the characteristics of squared error loss and absolute error loss, allowing for categorized discussion of experimental data. It accurately determines the loss value for the i-th data point in the target point cloud image for each case. This improves the accuracy of error analysis and effectively addresses the issue of a high number of outliers in the 3D point cloud image of the undischarged mash jar, significantly impacting the accuracy of the resulting center point coordinates.

[0083] S104, the processor sums the I data points of the target point cloud image to obtain a total loss value of the I data points of the target point cloud image.

[0084] Specifically, the calculation formula for the total loss value of I data points of the target point cloud image is as follows:

[0085]

[0086] in, is the total loss value of I data points of the target point cloud image, is the loss value of the i-th data point of the target point cloud image.

[0087] S105: The processor calculates the target gradient based on the total loss value of I data points of the target point cloud image.

[0088] Among them, the target gradient is the gradient of the total loss value of I data points in the target point cloud image.

[0089] Specifically, the target gradient represents the direction and rate of change of the total loss value of I data points of the target point cloud image. In multidimensional space, the target gradient is a vector, each component of which represents the total loss value of I data points of the target point cloud image relative to 、 、 、 The partial derivative of .

[0090] The calculation formula of the above target gradient is as follows:

[0091]

[0092] in, is the target gradient, is the total loss value of I data points of the target point cloud image, For the first jar to be used for the fermented mash The length of the semi-major axis of the cross section at the height, For the first jar to be used for the fermented mash The length of the semi-minor axis of the cross section at the height, For the first jar to be used for the fermented mash Height of the cross section center point abscissa, For the first jar to be used for the fermented mash Height of the center point of the cross section.

[0093] S106: The processor iterates the initial parameters of the target gradient to obtain target parameters.

[0094] Among them, the target parameters include the coordinates of the center of the ellipse model corresponding to the target point cloud image ( , ).

[0095] Specifically, the above target parameters also include: and , the initial parameters also include: and .

[0096] The calculation formula for the above target parameters is as follows:

[0097]

[0098] in, is the target parameter, is the number of iterations, For the t- Initial parameters after 1 iteration, For the t- The target gradient after 1 iteration, The learning rate is used to control the update amplitude of the initial parameters. Reasonable setting of the learning rate can accelerate the convergence of the initial parameters and obtain the optimal solution, that is, the target parameters, as soon as possible.

[0099] S107: The processor determines the position of the vat where the mash is to be discharged according to the N target parameters.

[0100] Specifically, the processor calculates the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images and the average value of the vertical coordinates of the center of the ellipse model corresponding to the N target point cloud images according to the following formula:

[0101]

[0102] in, is the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images, is the average value of the ordinate of the center of the ellipse model corresponding to the N target point cloud images, is the horizontal coordinate of the center of the ellipse model corresponding to the nth target point cloud image, is the vertical coordinate of the center of the ellipse model corresponding to the nth target point cloud image; according to the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images and the average value of the vertical coordinates of the center of the ellipse model corresponding to the N target point cloud images, determine the center coordinates of the plane where the ground vat to be discharged is located; the center coordinates of the plane where the ground vat to be discharged is located are determined as the position of the ground vat to be discharged.

[0103] From the above, it can be seen that according to the N target parameters, the coordinates of the center of the elliptical model corresponding to the target point cloud images at N heights can be obtained. In order to reduce the error, the processor obtains the center coordinates of the plane where the entire ground vat for the mash to be discharged is located, that is, the position of the ground vat for the mash to be discharged, based on the average value of the coordinates of the center of the elliptical model corresponding to the target point cloud images at N heights.

[0104] This setup accurately determines the location of the jar where the mash is to be dispensed. By processing data at multiple heights (the coordinates of the center of the elliptical model corresponding to the target point cloud image), it avoids inaccurate positioning results caused by a single data anomaly. Averaging the data provides robustness, resisting a certain degree of noise interference.

[0105] The present application provides a method for detecting the position of a jar to be discharged from a mash. In this method, a processor uses the Huber loss function. When experimental data is relatively close to outliers, the processor uses an absolute error loss calculation method to obtain the loss value of the i-th data point of the target point cloud image based on the residual value of the i-th data point of the target point cloud image. When experimental data is relatively close to outliers, the processor uses a squared error loss calculation method to obtain the loss value of the i-th data point of the target point cloud image based on the residual value of the i-th data point of the target point cloud image. Compared with the prior art method of fitting the center point coordinates of the three-dimensional point cloud image of the jar to be discharged from the mash using the least squares method and determining the position of the jar to be discharged from the jar based on the center point coordinates of the three-dimensional point cloud image of the jar to be discharged from the mash, this method has higher accuracy and can avoid the influence of excessive outlier data on the accuracy of the position determination of the jar to be discharged from the mash. This method facilitates the automatic handling equipment to accurately identify the position of the jar to be discharged from the jar when the mash is discharged and remove the mash from the jar. This method can flexibly select the calculation mode, reduce unnecessary data calculation processes, avoid generating redundant calculation data in the fitting process of the position of the mash jar, and thus avoid excessive occupation of processor memory resources. It has high pertinence and flexibility and saves fitting time.

[0106] The following describes how the processor obtains the target point cloud image. Figure 2 As shown, Figure 2This is a flow chart of the method for the processor to obtain the target point cloud image provided in the embodiment of the present application. The above step S101 specifically includes the following sub-steps:

[0107] S1011, the processor obtains a depth image of each ground cylinder and a color image of each ground cylinder.

[0108] Specifically, the above-mentioned processor obtains a depth image of each ground cylinder and a color image of each ground cylinder through TOF (Time of Flight Camera). TOF is a distance imaging camera system that uses time-of-flight technology. Before shooting, the internal and external parameters of TOF need to be calibrated so that the images obtained by the color lens and depth lens of TOF can be spatially aligned. Among them, the internal parameters are used to describe the internal imaging characteristics of TOF, such as focal length, principal point coordinates and distortion coefficient, which can be calibrated by a single TOF; the external parameters are used to describe the position and posture of TOF in three-dimensional space, such as rotation matrix and translation vector, which can be calibrated between multiple TOFs.

[0109] The TOF system emits a laser pulse towards each terrarium. The pulse then returns to the TOF system, using the speed of light and the time difference between the beam's round trip. The system then calculates the distance between each terrarium and the TOF system, thereby acquiring a depth image of each terrarium. The TOF system then captures a color image of each terrarium using an RGB (red, green, and blue) sensor.

[0110] S1012: The processor labels the color image of each ground cylinder to obtain a target color image of each ground cylinder.

[0111] Specifically, if Figure 3 As shown, Figure 3 Schematic diagram of the target color image of each ground vat provided in the embodiments of the present application. The target color image of each ground vat includes: a first annotated area 301 (a color image of the ground vat to be discharged with the annotated image) and a second annotated area 302 (a color image of the ground vat not to be discharged with the annotated image). It should be noted that in this step, the first annotated area 301 obtained by the staff using TOF to locate the location of the ground vat to be discharged and annotating the location of the ground vat to be discharged does not represent the exact location of the ground vat to be discharged.

[0112] The purpose of this setting is to facilitate the subsequent output of the mask image of the ground vat where the mash is to be produced through the neural network model based on the target color image of each ground vat.

[0113] S1013, the processor inputs the target color image of each ground vat into the trained neural network model, so that the neural network model outputs the mask image of the ground vat to be used for mash production.

[0114] Specifically, the neural network is a YOLOv7 model, which incorporates model reparameterization technology. By reorganizing the network structure and optimizing parameters, it effectively reduces the computational burden while maintaining high detection accuracy. The mask images of the jars to be prepared are binary or Boolean images of the same size as the target color image and depth image of each jar.

[0115] The above-mentioned YOLOv7 model has excellent target detection capabilities. The mask image of the ground jar where the mash is to be discharged obtained by the trained YOLOv7 model has high accuracy. In addition, this process does not require human intervention, which improves the efficiency of obtaining the mask image of the ground jar where the mash is to be discharged.

[0116] S1014: The processor performs AND operations on the pixel values ​​of the mask image of the ground vat to be used for producing mash and the pixel values ​​of the depth images of the J ground vats in sequence to obtain J first operation results.

[0117] S1015: If the j-th first operation result is 1, the depth image of the j-th ground vat is determined as the depth image of the ground vat for discharging mash.

[0118] S1016: If the j-th first operation result is 0, the depth image of the j-th ground vat is determined as the depth image of the ground vat not to be used for discharging mash.

[0119] Specifically, if Figure 4 As shown, Figure 4 This is a schematic diagram of an AND operation provided in an embodiment of the present application. The mask image of the above-mentioned ground vat to be used for mash production includes a first region of interest 501 and a first region of no interest 502. The depth images of the J ground vats include a third region of interest 603 and a third region of no interest 604. The pixel values ​​of the first region of interest 501 and the third region of interest 603 are both set to 1, and the pixel values ​​of the first region of no interest 502 and the third region of no interest 604 are both set to 0. The pixel values ​​of the mask image of the ground vat to be discharged (the pixel values ​​of the first area of ​​interest 501 and the first area of ​​no interest 502) are sequentially ANDed (compared) with the pixel values ​​of the depth images of the J ground vats (the pixel values ​​of the third area of ​​interest 603 and the third area of ​​no interest 604). If the j-th first operation result is 1, it means that the area where the depth image of the j-th ground vat is located is the third area of ​​interest 603, thereby determining that the depth image of the j-th ground vat is the depth image 605 of the ground vat to be discharged. If the j-th first operation result is 0, it means that the area where the depth image of the j-th ground vat is located is the third area of ​​no interest 604, thereby determining that the depth image of the j-th ground vat is the depth image 606 of the ground vat not to be discharged.

[0120] S1017: The processor performs AND operations on the pixel values ​​of the mask image of the ground vat to be used for producing mash and the pixel values ​​of the color images of the M ground vats in sequence to obtain M second operation results.

[0121] S1018: If the m-th second operation result is 1, the color image of the m-th ground vat is determined as the color image of the ground vat to be used for discharging mash.

[0122] S1019: If the m-th second operation result is 0, the color image of the m-th ground jar is determined as a color image of a ground jar that is not to be used for discharging mash.

[0123] Specifically, if Figure 5 As shown, Figure 5 Another AND operation diagram provided for an embodiment of the present application. The mask image of the above-mentioned ground vat to be used for mash production includes a first region of interest 501 and a first region of no interest 502. The color images of the R ground vats include a second region of interest 503 and a second region of no interest 504. The pixel values ​​of the first region of interest 501 and the second region of interest 503 are both set to 1, and the pixel values ​​of the first region of no interest 502 and the second region of no interest 504 are both set to 0. The pixel values ​​of the mask image of the ground vat to be discharged (the pixel values ​​of the first region of interest 501 and the first region of no interest 502) are sequentially ANDed (compared) with the pixel values ​​of the color images of R ground vats (the pixel values ​​of the second region of interest 503 and the second region of no interest 504). If the m-th second operation result is 1, it means that the area where the color image of the m-th ground vat is located is the second region of interest 503, thereby determining that the color image of the m-th ground vat is the color image 505 of the ground vat to be discharged. If the m-th second operation result is 0, it means that the area where the color image of the m-th ground vat is located is the second region of no interest 504, thereby determining that the color image of the m-th ground vat is the color image 506 of the ground vat not to be discharged.

[0124] S1020: The processor transforms the coordinates of the pixel points of the depth image of the earthenware vat from which the mash is to be discharged, and obtains three-dimensional space points corresponding to the pixel points of the depth image of the earthenware vat from which the mash is to be discharged.

[0125] Specifically, the calculation formula (coordinate conversion formula) for the three-dimensional space point corresponding to the pixel point of the depth image of the above-mentioned vat to be used for mash production is as follows:

[0126]

[0127] in, is the three-dimensional space point corresponding to the pixel point of the depth image of the jar where the mash is to be discharged. is the depth value of the depth image of the jar where the mash is to be discharged, is the horizontal coordinate of the focal length of TOF, is the focal length ordinate of TOF, is the horizontal coordinate of the principal point, is the vertical coordinate of the main point, u is the horizontal coordinate of the pixel point of the depth image of the jar where the mash is to be discharged, It is the vertical coordinate of the pixel point in the depth image of the vat where the mash is to be discharged.

[0128] S1021, the processor fuses the three-dimensional space points corresponding to the pixel points of the depth image of the earthen vat where the mash is to be discharged with the pixel points of the color image of the earthen vat where the mash is to be discharged, to obtain a three-dimensional point cloud image of the earthen vat where the mash is to be discharged.

[0129] Specifically, the processor assigns the color values ​​(R, G, B) of pixels extracted from the color image of the jar to be used for mash production to corresponding 3D spatial points in the depth image of the jar, thereby generating a complete 3D point cloud image of the jar. The 3D point cloud image of the jar is ultimately saved as a colored 3D point cloud format, such as PLY or PCD.

[0130] S1022: The processor cuts the three-dimensional point cloud image of the mash vat to be discharged in the horizontal direction to obtain a target point cloud image.

[0131] Specifically, the depth image obtained by TOF has a low resolution, and the depth value of the depth image is easily affected by noise, resulting in missing depth information in certain areas, which in turn causes holes of varying sizes to appear in the three-dimensional point cloud image of the jar where the mash is to be discharged. This not only affects the integrity of the three-dimensional point cloud image of the jar where the mash is to be discharged, but also creates certain difficulties for subsequent detection work.

[0132] Therefore, it is necessary to horizontally slice the 3D point cloud image of the jar where the mash is to be produced. The resulting 3D point cloud images (target point cloud images) of cross-sections of the jar at multiple heights are then analyzed to avoid holes in the 3D point cloud image of the jar, thereby ensuring the accuracy of the subsequently fitted position of the jar. Because the jar is a regular, cylindrical shape, the differences between the 3D point cloud images of cross-sections at multiple heights are minimal, facilitating the subsequent fitting of the jar's center coordinates (position).

[0133] In addition, since the TOF image cannot be completely parallel to the ground, there is a certain tilt angle between the image of the ground plane captured and the image of the real ground plane. Therefore, there is also a certain tilt angle between the point cloud image of the ideal ground plane generated from the captured ground plane image and the point cloud image of the real ground plane.

[0134] If the three-dimensional point cloud image of the mash jar is cut using the point cloud image of the plane where the ideal ground is located as the reference plane, there will be an inclination angle between the target point cloud image obtained by cutting and the point cloud image of the plane where the real ground is located, which will affect the subsequent determination of the coordinates of the center of the elliptical model corresponding to the N target point cloud images.

[0135] To further ensure the accuracy of the detection, the point cloud image of the plane where the ideal ground is located needs to be rotated to eliminate the tilt angle. The above-mentioned processor uses the RANSAC method to perform plane fitting on the point cloud image of the plane where the ideal ground is located to obtain the normal vector of the point cloud image of the plane where the ideal ground is located. Because the normal vector of the point cloud image of the plane where the real ground is located is (0,0,1), the point cloud image of the plane where the ideal ground is located needs to be rotated to coincide with the normal vector (0,0,1) of the point cloud image of the plane where the real ground is located. Among them, the basic principle of rotating the point cloud image of the plane where the ideal ground is located is to calculate the rotation matrix according to the Rodrigues rotation formula, so that the point cloud image of the plane where the ideal ground is located is rotated with the preset point as the center, and the rotation transformation is achieved through the rotation matrix.

[0136] In one embodiment, the processor obtains an initial depth image of each ground cylinder sent by the TOF camera; and filters the initial depth image of each ground cylinder through a Gaussian filter to obtain a depth image of each ground cylinder.

[0137] Specifically, there may be noise or errors in the initial depth image of each ground cylinder obtained by TOF, which will affect the subsequent fitting results. Therefore, it is necessary to filter out the noise of the initial depth image of each ground cylinder through a Gaussian filter to obtain a smooth initial depth image of each ground cylinder.

[0138] In this method, the Gaussian filter is a smoothing filter that preserves the edges of each floor cylinder's initial depth image. Therefore, while reducing noise in each floor cylinder's initial depth image, it also preserves important structural information. This method effectively reduces noise in each floor cylinder's initial depth image, resulting in a clearer and more accurate initial depth image.

[0139] On the basis of the above method embodiment, the embodiment of the present application further provides a position detection device for the vat to be used for discharging mash, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of a device for detecting the position of a mash vat to be discharged provided in an embodiment of the present application. The device includes: an acquisition module 61, a data processing module 62, a calculation module 63, and a detection module 64. The functions of each module are as follows:

[0140] An acquisition module 61 is used to acquire a target point cloud image; wherein the target point cloud image is a three-dimensional point cloud image of a cross section at the nth height of the base vat where the mash is to be discharged;

[0141] The data processing module 62 is used to input the coordinates of the i-th data point of the target point cloud image into the residual model to obtain the residual value of the i-th data point of the target point cloud image;

[0142] The calculation module 63 is used to calculate the loss value of the i-th data point of the target point cloud image based on the Huber loss function according to the residual value of the i-th data point of the target point cloud image:

[0143]

[0144] in, is the loss value of the i-th data point of the target point cloud image, is the residual value of the i-th data point of the target point cloud image, A threshold to control the smoothness of the loss function;

[0145] The calculation module 63 is further configured to sum I data points of the target point cloud image to obtain a total loss value of the I data points of the target point cloud image; and calculate a target gradient based on the total loss value of the I data points of the target point cloud image; wherein the target gradient is the gradient of the total loss value of the I data points of the target point cloud image;

[0146] The data processing module 62 is further configured to iterate the initial parameters of the target gradient to obtain target parameters; wherein the target parameters include the coordinates of the center of the ellipse model corresponding to the target point cloud image;

[0147] The detection module 64 is used to determine the position of the vat where the mash is to be discharged based on N target parameters.

[0148] Preferably, the acquisition module 61 is specifically used to acquire the depth image of each ground cylinder and the color image of each ground cylinder; mark the color image of each ground cylinder to obtain the target color image of each ground cylinder; input the target color image of each ground cylinder into the trained neural network model, so that the neural network model outputs the mask image of the ground cylinder to be discharged; the pixel values ​​of the mask image of the ground cylinder to be discharged are sequentially and operated with the pixel values ​​of the depth images of J ground cylinders to obtain J first operation results; if the jth first operation result is 1, the depth image of the jth ground cylinder is determined as the depth image of the ground cylinder to be discharged; the pixel values ​​of the mask image of the ground cylinder to be discharged are sequentially and operated with the pixel values ​​of the color images of M ground cylinders to obtain M second Operation result; if the m-th second operation result is 1, the color image of the m-th ground cylinder is determined as the color image of the ground cylinder to be discharged; the coordinates of the pixel points of the depth image of the ground cylinder to be discharged are transformed to obtain the three-dimensional space points corresponding to the pixel points of the depth image of the ground cylinder to be discharged; the three-dimensional space points corresponding to the pixel points of the depth image of the ground cylinder to be discharged are merged with the pixel points of the color image of the ground cylinder to be discharged to obtain the three-dimensional point cloud image of the ground cylinder to be discharged; the three-dimensional space points corresponding to the pixel points of the depth image of the ground cylinder to be discharged are merged with the pixel points of the color image of the ground cylinder to be discharged to obtain the three-dimensional point cloud image of the ground cylinder to be discharged; the three-dimensional point cloud image of the ground cylinder to be discharged is cut along the horizontal direction to obtain the target point cloud image.

[0149] Preferably, the acquisition module 61 is further configured to determine the depth image of the j-th ground vat as a depth image of a ground vat not to be used for discharging mash if the j-th first operation result is 0.

[0150] Preferably, the acquisition module 61 is further configured to determine the color image of the mth ground vat as a color image of a ground vat not to be used for discharging mash if the mth second operation result is 0.

[0151] Preferably, the detection module 64 is specifically configured to calculate the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images and the average value of the vertical coordinates of the center of the ellipse model corresponding to the N target point cloud images according to the following formula:

[0152]

[0153] in, is the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images, is the average value of the ordinate of the center of the ellipse model corresponding to the N target point cloud images, is the horizontal coordinate of the center of the ellipse model corresponding to the nth target point cloud image, is the ordinate of the center of the elliptical model corresponding to the nth target point cloud image; according to the average value of the abscissa of the center of the elliptical model corresponding to the N target point cloud images and the average value of the ordinate of the center of the elliptical model corresponding to the N target point cloud images, determine the coordinates of the center of the elliptical model corresponding to the N target point cloud images; the coordinates of the center of the elliptical model corresponding to the N target point cloud images are determined as the position of the ground jar to be discharged from the mash.

[0154] Preferably, the acquisition module 61 is further configured to acquire an initial depth image of each ground cylinder sent by the TOF camera;

[0155] The data processing module 62 is further configured to filter the initial depth image of each ground cylinder through a Gaussian filter to obtain a depth image of each ground cylinder.

[0156] The position detection device for the ground vat for discharging mash provided in the embodiment of the present application has the same technical features as the position detection method for the ground vat for discharging mash provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0157] The present application also provides a computing device. Figure 7 As shown, this figure is a schematic diagram of a computing device provided by an embodiment of the present application, wherein the computing device 400 includes a bus 401, a processor 402, a communication interface 403, and a memory 404. The processor 402, the memory 404, and the communication interface 403 communicate with each other via the bus 401.

[0158] The bus 401 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0159] The processor 402 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0160] Communication interface 403 is used for external communication. Memory 404 may include volatile memory, such as random access memory (RAM). Memory 404 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0161] The memory 404 stores executable codes, and the processor 402 executes the executable codes to perform the aforementioned method for detecting the position of the vat where the mash is to be discharged.

[0162] Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, or magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the aforementioned method for detecting the position of the vat where mash to be discharged is to be dispensed.

[0163] The present application also provides a computer program product comprising one or more computer instructions that, when loaded and executed on a computing device, fully or partially generate the process or function described in the present application.

[0164] The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0165] When the computer program product is executed by a computer, the computer executes any of the aforementioned methods for detecting the position of a vat for discharging mash. The computer program product may be a software installation package, which can be downloaded and executed on a computer when any of the aforementioned methods for detecting the position of a vat for discharging mash is required.

[0166] The descriptions of the processes or structures corresponding to the above figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0167] The above description is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting the position of a vat for discharging mash, characterized in that: Applied to a processor, the method includes: The processor obtains a target point cloud image; wherein the target point cloud image is a three-dimensional point cloud image of a cross section at an n-th height of the jar where the mash is to be discharged; The processor inputs the coordinates of the i-th data point of the target point cloud image into the residual model to obtain the residual value of the i-th data point of the target point cloud image; The processor calculates the loss value of the i-th data point of the target point cloud image based on the Huber loss function and the residual value of the i-th data point of the target point cloud image: in, is the loss value of the i-th data point of the target point cloud image, is the residual value of the i-th data point of the target point cloud image, A threshold to control the smoothness of the loss function; The processor sums I data points of the target point cloud image to obtain a total loss value of the I data points of the target point cloud image; The processor calculates a target gradient based on the total loss value of I data points of the target point cloud image; wherein the target gradient is the gradient of the total loss value of I data points of the target point cloud image; The processor iterates the initial parameters of the target gradient to obtain target parameters; wherein the target parameters include the coordinates of the center of the ellipse model corresponding to the target point cloud image; The processor determines the position of the vat where the mash is to be discharged according to N target parameters.

2. The method for detecting the position of the jar for discharging mash according to claim 1, characterized in that: The step of the processor acquiring the target point cloud image includes: The processor acquires a depth image of each ground cylinder and a color image of each ground cylinder; The processor labels the color image of each ground cylinder to obtain a target color image of each ground cylinder; The processor inputs the target color image of each ground vat into the trained neural network model, so that the neural network model outputs the mask image of the ground vat to be used for mash production; The processor performs AND operations on pixel values ​​of the mask image of the ground vat to be discharged and pixel values ​​of the depth images of J ground vats in sequence to obtain J first operation results; If the j-th first operation result is 1, the processor determines the depth image of the j-th ground vat as the depth image of the ground vat to be used for discharging mash; The processor performs an AND operation on the pixel values ​​of the mask image of the ground vat to be discharged and the pixel values ​​of the color images of the M ground vats in sequence to obtain M second operation results; If the mth second operation result is 1, the processor determines the color image of the mth ground vat as the color image of the ground vat to be used for discharging mash; The processor transforms the coordinates of the pixel points of the depth image of the vat to be discharged, to obtain three-dimensional space points corresponding to the pixel points of the depth image of the vat to be discharged; The processor fuses the three-dimensional space points corresponding to the pixel points of the depth image of the jar to be discharged from the mash with the pixel points of the color image of the jar to be discharged from the mash to obtain a three-dimensional point cloud image of the jar to be discharged from the mash; The processor cuts the three-dimensional point cloud image of the vat to be used for discharging mash along a horizontal direction to obtain the target point cloud image.

3. The method for detecting the position of the jar for discharging mash according to claim 2, characterized in that: The method further comprises: If the j-th first operation result is 0, the processor determines the depth image of the j-th ground vat as a depth image of a ground vat not to be used for discharging mash.

4. The method for detecting the position of the jar for discharging mash according to claim 2, wherein: The method further comprises: If the mth second operation result is 0, the processor determines the color image of the mth ground vat as a color image of a ground vat that is not to be used for discharging mash.

5. The method for detecting the position of the jar for discharging mash according to claim 1, wherein: The step of the processor determining the position of the vat to be used for discharging the mash according to N target parameters includes: The processor calculates the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images and the average value of the vertical coordinates of the center of the ellipse model corresponding to the N target point cloud images according to the following formula: in, is the average value of the horizontal coordinates of the center of the ellipse model corresponding to the N target point cloud images, is the average value of the ordinate of the center of the ellipse model corresponding to the N target point cloud images, is the horizontal coordinate of the center of the ellipse model corresponding to the nth target point cloud image, is the ordinate of the center of the ellipse model corresponding to the nth target point cloud image; The processor determines the coordinates of the center of the ellipse model corresponding to the N target point cloud images according to the average value of the abscissa of the center of the ellipse model corresponding to the N target point cloud images and the average value of the ordinate of the center of the ellipse model corresponding to the N target point cloud images; The processor determines the coordinates of the center of the ellipse model corresponding to the N target point cloud images as the position of the vat where the mash is to be discharged.

6. The method for detecting the position of the jar for discharging mash according to claim 2, characterized in that: The method further comprises: The processor obtains an initial depth image of each ground cylinder sent by the TOF camera; The processor filters the initial depth image of each ground cylinder through a Gaussian filter to obtain a depth image of each ground cylinder.

7. A device for detecting the position of a vat for discharging mash, characterized in that: The device comprises: An acquisition module is used to acquire a target point cloud image; wherein the target point cloud image is a three-dimensional point cloud image of a cross section at the nth height of the base vat where the mash is to be discharged; a data processing module, configured to input the coordinates of the i-th data point of the target point cloud image into a residual model to obtain a residual value of the i-th data point of the target point cloud image; A calculation module is used to calculate the loss value of the i-th data point of the target point cloud image based on the Huber loss function according to the residual value of the i-th data point of the target point cloud image: in, is the loss value of the i-th data point of the target point cloud image, is the residual value of the i-th data point of the target point cloud image, A threshold to control the smoothness of the loss function; The calculation module is further configured to sum I data points of the target point cloud image to obtain a total loss value of the I data points of the target point cloud image; and calculate a target gradient based on the total loss value of the I data points of the target point cloud image; wherein the target gradient is the gradient of the total loss value of the I data points of the target point cloud image; The data processing module is further configured to iterate the initial parameters of the target gradient to obtain target parameters; wherein the target parameters include the coordinates of the center of the ellipse model corresponding to the target point cloud image; The detection module is used to determine the position of the vat for discharging the mash according to N target parameters.

8. A computing device, characterized in that including memory and processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program. When the computer program is executed by a computer, the computer performs the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises one or more computer instructions. When the computer instructions are executed by a computer, the computer performs the method according to any one of claims 1 to 6.

Images