Weight detection method, device, system, electronic device and storage medium

Abstract

The present application provides a kind of weight detection method, device, system, electronic equipment and storage medium, wherein the method comprises: obtaining the image of the object to be detected under at least two perspectives;Feature point matching is carried out on feature points in the image under at least two perspectives, to obtain feature point group, feature point group includes feature points under at least two perspectives;Based on feature point group, point cloud reconstruction is carried out on the object to be detected, to obtain the point cloud data of the object to be detected;Point cloud data is matched with template point cloud data, and the weight of the template point cloud data matched with the point cloud data is determined as the weight of the object to be detected.The present application provides a kind of weight detection method, device, system, electronic equipment and storage medium, which can realize non-contact, automatic intelligent weight detection of the object to be detected, thereby avoiding the problem caused by manual weighing in the prior art, not only improves the efficiency of weight detection, reduces labor cost, but also improves accuracy and reliability.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a weight detection method, apparatus, system, electronic device, and storage medium. Background Technology

[0002] Intelligent dead pig weight estimation algorithms play a crucial role for insurance companies, especially in dead pig compensation, where falsely reporting the length and weight of the dead pigs is one of the major reasons for insurance companies to pay out more claims.

[0003] In existing technologies, there are two main methods for estimating the weight of dead pigs: expert experience and weighbridge weighing. Expert experience: Experts estimate the weight of the dead pig based on information such as the age and breed of the pig, and on their past experience with the pigs. Weighbridge weighing: Farmers place the pig flat on a weighbridge and take the weighbridge reading as the weight of the dead pig.

[0004] Expert experience, acquired through human intervention, involves subjective judgment and decision-making, neglecting individual differences and thus prone to overestimating or underestimating weights. Even when using weighbridges, the weighing of dead pigs is primarily used in hog insurance claims, making weighbridges susceptible to manipulation and unreliable data, potentially leading to insurance fraud. Furthermore, weighbridges and similar equipment require frequent calibration, increasing labor costs and operational complexity. Summary of the Invention

[0005] This invention provides a weight detection method, device, system, electronic device, and storage medium to solve the defects of inaccurate and unreliable weight estimation of dead pigs, as well as the complexity and time-consuming nature of operation in the prior art.

[0006] This invention provides a weight detection method, comprising:

[0007] Acquire images of the object to be detected from at least two viewpoints;

[0008] Feature point matching is performed on feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints.

[0009] Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain the point cloud data of the object to be detected;

[0010] The point cloud data is matched with the template point cloud data, and the weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected.

[0011] According to the weight detection method provided by the present invention, the step of performing feature point matching on feature points in the images from at least two viewpoints to obtain a feature point group includes:

[0012] Based on the shooting angle of the images from each viewpoint, the feature point patches in the images from each viewpoint are spatially transformed, and the features of the spatially transformed feature point patches are extracted to obtain the feature point description information of each viewpoint in the same space.

[0013] Based on the feature point description information of the at least two viewpoints, feature point matching is performed on the feature points in the images under the at least two viewpoints to obtain feature point groups.

[0014] According to the weight detection method provided by the present invention, the step of determining the shooting angle includes:

[0015] Based on the feature point patches where the feature points are located in the images from each viewpoint, the orientation of the images from each viewpoint is estimated to obtain the shooting angle from each viewpoint.

[0016] According to the weight detection method provided by the present invention, the step of estimating the orientation of the image from each viewpoint based on the feature point patch where the feature points in the image are located, to obtain the shooting angle from each viewpoint, includes:

[0017] Based on the orientation estimation model, the orientation of the images under each viewpoint is estimated by applying the feature point patch where the feature points are located in the images under each viewpoint, and the shooting angle under each viewpoint is obtained.

[0018] The orientation estimation model is trained based on the patch of sample 3D points under different viewpoints, with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point under different viewpoints.

[0019] According to the weight detection method provided by the present invention, the step of extracting features from the spatially transformed feature point patch to obtain feature point description information of each viewpoint in the same space includes:

[0020] Based on the feature extraction model, the feature point map patch after spatial transformation is applied to extract features from the feature point map patch to obtain the feature point description information of the feature point map patch.

[0021] The feature point description information of the feature point patch is mapped to the image of each viewpoint to which the feature point patch belongs, so as to obtain the feature point description information of each viewpoint in the same space;

[0022] The feature extraction model is trained based on the similarity between feature point patches containing matching feature points in sample images from at least two viewpoints, and / or the similarity between feature point patches containing non-matching feature points.

[0023] According to the weight detection method provided by the present invention, the step of matching the point cloud data with the template point cloud data includes:

[0024] When the object to be detected is livestock, the body length of the object to be detected is determined based on the point cloud data;

[0025] Based on the body length of the object to be detected, first template point cloud data matching the body length is obtained by filtering from the template point cloud data;

[0026] The point cloud data is matched with the first template point cloud data.

[0027] According to the weight detection method provided by the present invention, the step of reconstructing the point cloud of the object to be detected based on the feature point group to obtain the point cloud data of the object to be detected includes:

[0028] Based on the feature point group, the intrinsic and extrinsic parameters of the image acquisition device and its pose at each viewpoint are determined, and point cloud reconstruction is performed on the feature points in the images at least two viewpoints to obtain the initial point cloud data of the object to be detected.

[0029] Based on the three-dimensional points in the initial point cloud data and the pose of the image acquisition device at the reference viewpoint, the initial point cloud data is expanded to obtain the point cloud data of the object to be detected.

[0030] The present invention also provides a weight detection device, comprising:

[0031] An image acquisition unit is used to acquire images of the object to be detected from at least two viewpoints.

[0032] A feature point matching unit is used to perform feature point matching on feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints.

[0033] The point cloud reconstruction unit is used to reconstruct the point cloud of the object to be detected based on the feature point group, so as to obtain the point cloud data of the object to be detected.

[0034] The point cloud matching unit is used to match the point cloud data with template point cloud data, and determine the weight of the template point cloud data that matches the point cloud data as the weight of the object to be detected.

[0035] The present invention also provides a weight detection system, comprising: an image acquisition device and a weight detection device;

[0036] The image acquisition device is used to acquire images of the object to be detected from at least two perspectives, and to send the images from the at least two perspectives to the weight detection device;

[0037] The weight detection device is used to perform feature point matching on feature points in the images from at least two viewpoints to obtain a feature point group. Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain point cloud data of the object to be detected. The point cloud data is then matched with template point cloud data. The weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected. The feature point group includes feature points from at least two viewpoints.

[0038] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the weight detection method as described above.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the weight detection method as described above.

[0040] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the weight detection method as described above.

[0041] The weight detection method, apparatus, system, electronic device, and storage medium provided by this invention extract feature points from images from at least two perspectives and match these feature points between the images to obtain feature point groups. Point cloud reconstruction is then performed on the object to be detected using these feature point groups to obtain point cloud data. The reconstructed point cloud data is then matched with template point cloud data in a template database; the weight corresponding to the successfully matched template point cloud data is the weight of the object to be detected. This method enables contactless, automated, and intelligent weight detection of the object, avoiding the problems associated with manual weighing in existing technologies. It not only improves the efficiency of weight detection and reduces labor costs but also enhances accuracy and reliability. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 This is one of the flowcharts of the weight detection method provided by the present invention;

[0044] Figure 2 This is one of the flowcharts for step 120 in the weight detection method provided by the present invention;

[0045] Figure 3 This is a flowchart illustrating the feature point map acquisition method provided by the present invention;

[0046] Figure 4 This is the second flowchart of step 120 in the weight detection method provided by the present invention;

[0047] Figure 5 This is a flowchart illustrating step 140 in the weight detection method provided by the present invention;

[0048] Figure 6 This is a flowchart illustrating step 130 in the weight detection method provided by the present invention;

[0049] Figure 7 This is the second schematic flowchart of the weight detection method provided by the present invention;

[0050] Figure 8 This is a schematic diagram of the weight detection device provided by the present invention;

[0051] Figure 9 This is a schematic diagram of the weight detection system provided by the present invention;

[0052] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] Currently, there are two main methods for estimating the weight of dead pigs: expert experience and weighbridge weighing. Expert experience relies on human judgment and decision-making, neglecting individual differences and thus easily leading to overestimation or underestimation. Weighbridge weighing is susceptible to tampering, resulting in unreliable data. Furthermore, weighbridges require frequent calibration, increasing labor costs and operational complexity.

[0055] In response to the above situation, this invention provides a weight detection method that can be applied to the weight detection of dead pigs in the context of pig insurance claims; it can also be applied to the weight detection of other livestock such as cattle and sheep or poultry such as chickens and ducks. At the same time, this method does not specifically limit whether the object to be detected has vital signs, that is, it can be used to detect the weight of both dead pigs and live pigs.

[0056] The weight detection method provided in this invention first acquires images of the object to be detected from at least two perspectives. Feature points are extracted from these images from each perspective using an image processing algorithm, and feature point matching is performed between the images from each perspective to obtain feature point groups. Point cloud reconstruction is then performed on the object to be detected using these feature point groups to obtain point cloud data. The reconstructed point cloud data is then matched with template point cloud data in a template database. The weight corresponding to the successfully matched template point cloud data is the weight of the object to be detected. This method enables contactless, automated, and intelligent weight detection of the object, avoiding the problems associated with manual weighing in existing technologies. It not only improves the efficiency of weight detection and reduces labor costs but also enhances accuracy and reliability.

[0057] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0058] Figure 1 This is one of the flowcharts illustrating the weight detection method provided by the present invention. The execution entity for each step in this method can be a weight detection device, which can be implemented through software and / or hardware. This device can be integrated into an electronic device, such as a personal computer, a cloud device, or a mobile device like a smartphone or tablet. Figure 1 As shown, the weight detection method provided in this embodiment of the invention may include the following steps:

[0059] Step 110: Obtain images of the object to be detected from at least two viewpoints.

[0060] Specifically, the object to be tested is the object that needs to be weighed. The object to be tested can be livestock such as cattle and sheep or poultry such as chickens and ducks, such as dead pigs, dead cows, or live pigs.

[0061] Image acquisition devices can capture images of the object to be detected from at least two angles, obtaining images from at least two perspectives; alternatively, they can extract image frames from video from at least two angles. For example, first, an image containing the object to be detected can be captured from any angle. Then, the shooting angle of the image acquisition device can be adjusted to capture another image containing the object to be detected, and so on, thus obtaining images from at least two perspectives. These images from at least two perspectives can characterize the morphological features of the same object to be detected from different perspectives.

[0062] Furthermore, the images from at least two perspectives can be acquired by the same image acquisition device or by different image acquisition devices. If the images were acquired by different image acquisition devices, preprocessing can be performed on the images from at least two perspectives. This includes adjusting image resolution, standardizing image size, or correcting image distortion.

[0063] Step 120: Perform feature point matching on feature points in images from at least two viewpoints to obtain feature point groups, which include feature points from at least two viewpoints.

[0064] Specifically, feature points in images from at least two perspectives refer to relatively prominent points in the image, such as contour points, bright spots in darker areas, and dark spots in brighter areas; they can also be key points, such as key points of key parts of a pig, such as its ears, nose, and rump.

[0065] To obtain feature points in an image, feature point extraction algorithms can be used. Commonly used algorithms, such as ORB (Oriented Fast and Rotated BRIEF) and SIFT (Scale Invariant Feature Transform), can be employed for coarse feature extraction. Alternatively, pre-trained feature point extraction models can be used. The extracted feature points reflect their position, image features, and semantic features from their respective viewpoints.

[0066] After obtaining the feature points in the image from each viewpoint, feature point matching can be performed between these viewpoints to obtain feature point groups. Each matched feature point group includes feature points from at least two viewpoints. It is understandable that the smaller the difference between feature points from different viewpoints, the greater the probability that the feature point belongs to the same region of the object to be detected, and thus the greater the probability that feature points from different viewpoints can be matched into a single feature point group; conversely, the greater the difference between feature points from different viewpoints, the lower the probability that the feature point belongs to the same region of the object to be detected, and thus the lower the probability that feature points from different viewpoints can be matched into a single feature point group.

[0067] For example, image A taken at a 45-degree angle includes four feature points: a, b, c, and d. Image B taken at a 90-degree angle includes five feature points: e, f, g, h, and k. After feature point matching, a and f are found to be matched, and c and h are found to be matched, resulting in two sets of feature points. The first set includes feature points a and f; the second set includes feature points c and h.

[0068] For example, consider an image C taken at a 120-degree angle, containing three feature points: o, p, and k. Feature point matching reveals that feature points o, a, and f are matched, resulting in two sets of feature points. The first set includes feature points o, a, and f; the second set includes feature points c and h.

[0069] Step 130: Based on the feature point group, perform point cloud reconstruction on the object to be detected to obtain the point cloud data of the object to be detected.

[0070] Specifically, since feature point groups can reflect the matching relationship of the same feature point of the object to be detected in at least two viewpoints, based on the feature point groups, the intrinsic and extrinsic parameters of the image acquisition device can be estimated by image deep learning methods, thereby reconstructing the point cloud of the object to be detected. The resulting point cloud data can show the structural features of the object to be detected in space from the perspective of three-dimensional space.

[0071] First, sparse point cloud data of the object to be detected can be reconstructed based on the feature point group. Then, depth data can be used to expand the sparse point cloud data to obtain dense point cloud data of the object to be detected.

[0072] Step 140: Match the point cloud data with the template point cloud data, and determine the weight of the template point cloud data that matches the point cloud data as the weight of the object to be detected.

[0073] Specifically, after obtaining the point cloud data of the object to be detected, the point cloud data can be matched with the template point cloud data in the template library. The matching is performed by calculating the registration loss to obtain the template point cloud data that matches the point cloud data. That is, the registration loss between the obtained template point cloud data and the object to be detected is minimized, so the weight of the obtained template point cloud data can be determined as the weight of the object to be detected.

[0074] The template point cloud data can be pre-stored, obtained by 3D scanning of a real object, or obtained by computer simulation. This embodiment of the invention does not specifically limit the specifics.

[0075] In specific matching, point cloud data can be directly matched with template point cloud data, or the template point cloud data can be filtered first, and then the point cloud data can be matched with the filtered template point cloud data, thereby improving the matching efficiency of point cloud data.

[0076] The method provided in this invention extracts feature points from images from at least two perspectives and matches these feature points between the images to obtain feature point groups. It then reconstructs a point cloud from these feature point groups to obtain point cloud data. Finally, it matches the reconstructed point cloud data with template point cloud data in a template database; the weight corresponding to the successfully matched template point cloud data is then taken as the weight of the object to be detected. This method enables contactless, automated, and intelligent weight detection of the object, avoiding the problems associated with manual weighing in existing technologies. It not only improves the efficiency of weight detection and reduces labor costs but also enhances accuracy and reliability.

[0077] Based on the above embodiments, Figure 2 This is one of the flowcharts illustrating step 120 in the weight detection method provided by this invention, as shown below. Figure 2 As shown, step 120 specifically includes:

[0078] Step 121: Based on the shooting angle of the images from each viewpoint, perform spatial transformation on the feature point patches where the feature points are located in the images from each viewpoint, and extract features from the spatially transformed feature point patches to obtain feature point description information for each viewpoint in the same space.

[0079] Step 122: Based on the feature point description information from at least two viewpoints, perform feature point matching on the feature points in the images from at least two viewpoints to obtain feature point groups.

[0080] Specifically, to achieve feature point matching and obtain feature point groups, feature point matching can be performed based on feature point description information from different viewpoints in the same space. Feature point description information can accurately describe the features of the feature point patch where each feature point is located. By comparing the similarity of feature point description information, it is determined whether a match can be successfully obtained, thus obtaining a feature point group.

[0081] The feature point patch where each feature point is located can be obtained by detecting multiple potential feature points in the image from each viewpoint, and then cutting these potential feature points into smaller patches.

[0082] Considering that the feature point patches are obtained from different perspectives, before describing the feature points, spatial transformation can be performed on the feature point patches in the images from each perspective to transform them into the same space.

[0083] Preferably, the feature point patches containing the feature points in the images from different viewpoints can also be scaled to the same scale.

[0084] For feature point patches transformed into the same space, feature extraction, or feature description, can be performed on the feature point patches to obtain feature point description information from different perspectives in the same space.

[0085] Then, step 122 is executed, where feature point matching is performed on feature points in images from at least two different viewpoints by comparing feature point description information from each viewpoint, resulting in feature point groups. Specifically, similarity matching of feature point description information can be performed by calculating the distance between feature point description information from each viewpoint. The closer the distance between the two, the higher the matching degree; conversely, the farther the distance between the two, the lower the matching degree.

[0086] The method provided in this invention improves the accuracy and reliability of feature point matching by performing spatial transformation on the feature point patches in the image from each viewpoint to obtain feature point description information in the same space from each viewpoint, and then performing feature point matching based on the feature point description information.

[0087] Based on any of the above embodiments, the step of determining the shooting angle includes:

[0088] Based on the feature point patches where the feature points are located in the images from each viewpoint, the orientation of the images from each viewpoint is estimated to obtain the shooting angle from each viewpoint.

[0089] Specifically, the shooting angle of the image under each viewpoint can be realized based on the feature point patch where the feature point is located in the image under each viewpoint. After the feature point is detected, the image under each viewpoint is segmented with the feature point as the center to obtain the feature point patch where the feature point is located.

[0090] In one embodiment, the feature point patch containing the feature point can be implemented using a twin feature pyramid model. Figure 3 This is a flowchart illustrating the feature point patch acquisition method provided by the present invention, as shown below. Figure 3 As shown, images from two different viewpoints are input into two sub-twin branches of the twin feature pyramid model to obtain a multi-scale depth feature pyramid for each viewpoint. Then, softmax is used to obtain multiple latent feature points in each viewpoint image. These latent feature points are then cut into small patches, and NMS (Non-Maximum Suppression) is used at multiple scales to obtain feature point patches at a single scale.

[0091] Based on this, the orientation of each feature point patch is estimated. It can be understood that the orientation of each feature point patch in an image belonging to the same viewpoint is consistent, while the orientation of each feature point patch in an image not belonging to the same viewpoint is inconsistent. Based on the orientation of each feature point patch, the shooting angle of the image under each viewpoint can be obtained.

[0092] Based on any of the above embodiments, based on the feature point patches where the feature points in the images from each viewpoint are located, orientation estimation is performed on the images from each viewpoint to obtain the shooting angle from each viewpoint, including:

[0093] Based on the orientation estimation model, the orientation of the images under each viewpoint is estimated by applying the feature point patch where the feature points are located in the images under each viewpoint, and the shooting angle under each viewpoint is obtained.

[0094] The orientation estimation model is trained based on the patch of sample 3D points under different viewpoints, with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point under different viewpoints.

[0095] Specifically, method estimation for images from various viewpoints can be achieved through a orientation estimation model. This orientation estimation model can be a CNN feature extractor. Feature point patches containing feature points from each viewpoint are input into the trained orientation estimation model. The trained model then performs method estimation on each feature point patch, obtaining the rotation angle output by the orientation estimation model. This rotation angle represents the shooting angle from each viewpoint. Subsequently, the feature point patches are rotated according to this rotation angle to obtain feature point patches in the same spatial plane.

[0096] The orientation estimation model can be trained through the following steps: collect a large number of map tiles from different perspectives, and mark the matching feature points in the map tiles from different perspectives as sample 3D points. The initial orientation estimation model is trained with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point from different perspectives.

[0097] It should be noted that the feature point description information here can be obtained by extracting features from the feature point patch using a pre-trained feature extraction model and calculating the feature point description information. Based on the pre-trained feature extraction model, the initial orientation estimation model is trained to obtain a trained orientation estimation model, which can then regress the shooting angle from each viewpoint.

[0098] The method provided in this embodiment of the invention trains an initial orientation estimation model based on a pre-trained feature extraction model and on the tiles of sample 3D points under different viewpoints, with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point under different viewpoints. The orientation estimation model trained in this way can achieve more accurate orientation estimation.

[0099] Based on any of the above embodiments Figure 4 This is the second flowchart of step 120 in the weight detection method provided by the present invention, as shown below. Figure 4 As shown, in step 120, feature extraction is performed on the feature point patch after spatial transformation to obtain feature point description information for each viewpoint in the same space, specifically including:

[0100] Step 123: Based on the feature extraction model, apply the spatially transformed feature point map to extract features from the feature point map to obtain the feature point description information of the feature point map.

[0101] Step 124: Map the feature point description information of the feature point patch to the image of each viewpoint to which the feature point patch belongs, and obtain the feature point description information of each viewpoint in the same space.

[0102] The feature extraction model is trained based on the similarity between feature point patches containing matching feature points in sample images from at least two viewpoints, and / or the similarity between feature point patches containing non-matching feature points.

[0103] Specifically, a spatial transformation is performed on the feature point patches containing feature points in the image from each viewpoint. After obtaining the spatially transformed feature point patches, these patches are input into a feature extraction model to extract features, resulting in feature point description information output by the model. This feature extraction model can specifically be a feature descriptor.

[0104] After obtaining the feature point description information of the feature point map, the feature point description information of each feature point map is mapped to the image under the viewpoint to which the feature point map belongs. At the same time, the feature point description information of feature point map blocks belonging to the same viewpoint can be merged to obtain the feature point description information of each viewpoint in the same space.

[0105] The feature extraction model is trained based on the following steps: acquiring a large number of sample images from at least two perspectives and labeling matching feature points on the sample images; inputting the sample images into the feature extraction branches of the initial feature extraction model, with each feature extraction branch sharing weights.

[0106] During training, the initial feature extraction model can learn the commonalities between feature point patches containing matching feature points, and at the same time learn the differences between feature point patches containing non-matching feature points. As a result, the feature extraction model trained in this way can extract more accurate feature point description information.

[0107] The similarity between feature point patches can be calculated using the distance between them. Matching feature points are those marked in the sample images from one viewpoint that match those in the sample images from another viewpoint; conversely, they are mismatching feature points.

[0108] In one embodiment, the loss function of the feature extraction model can be Contrastive Loss. The distance between the feature descriptions of matching feature points is minimized, while the distance between the feature descriptions of mismatched feature points is greater than a preset distance threshold (margin). Contrastive Loss simultaneously constrains the intra-class compactness of matching feature points and the inter-class separation of mismatched feature points. The loss function of the feature extraction model can be expressed in the following form:

[0109]

[0110] In the formula, f i f represents the feature description information of the feature point patch in the image from the i-th viewpoint; j This represents the feature description information of the feature point patch in the image from the j-th viewpoint; y ij =1 indicates that the feature point in the image from the i-th viewpoint matches the feature point in the image from the j-th viewpoint; y ij =-1 indicates that the feature points in the image from the i-th viewpoint are not matching the feature points in the image from the j-th viewpoint, and m represents the preset distance threshold.

[0111] The method provided in this invention uses sample images from at least two viewpoints that are labeled with matching feature points to train an initial feature extraction model based on the similarity between feature point patches containing matching feature points and / or the similarity between feature point patches containing non-matching feature points. The feature extraction model trained in this way can accurately describe the features of the feature point patches containing each feature point, and obtain a more accurate feature description vector, thereby achieving more accurate feature point matching.

[0112] Based on any of the above embodiments Figure 5 This is a flowchart illustrating step 140 of the weight detection method provided by the present invention, as shown below. Figure 5 As shown, step 140 involves matching the point cloud data with the template point cloud data, specifically including:

[0113] Step 141: When the object to be detected is livestock, determine the body length of the object to be detected based on point cloud data;

[0114] Step 142: Based on the body length of the object to be detected, select the first template point cloud data that matches the body length from the template point cloud data;

[0115] Step 143: Match the point cloud data with the first template point cloud data.

[0116] Specifically, when the object to be detected is livestock, such as pigs, cattle, and sheep, a two-stage matching method can be used to achieve the matching between point cloud data and template point cloud data. First, the first template point cloud data is obtained by filtering from the template point cloud data based on the body length of the object to be detected, and then the point cloud data is matched with the first template point cloud data.

[0117] First, based on the point cloud data of the object to be detected, the length from the head to the tail root is calculated as the body length. The first template point cloud data is then obtained by filtering based on the body length.

[0118] Within the obtained first template point cloud data, the point cloud data of the object to be detected is registered with the point cloud in the first template point cloud data. The registration loss between the first template point cloud data and the point cloud data is calculated respectively. The chamber loss function can be used to minimize the registration loss. The first template point cloud data with the smallest registration loss with the point cloud data is taken as the matching template point cloud data. The weight corresponding to the matching template point cloud data is taken as the weight of the object to be detected.

[0119] Preferably, the head data in the point cloud data can be removed to obtain point cloud data containing only the body, and then the point cloud data containing only the body can be matched with the first template point cloud data.

[0120] The method provided in this embodiment of the invention first determines the body length of the object to be detected, then filters the first template point cloud data from the template point cloud data based on the body length, and then matches the point cloud data with the first template point cloud data, thereby improving the efficiency and accuracy of point cloud data matching.

[0121] Based on any of the above embodiments Figure 6 This is a flowchart illustrating step 130 of the weight detection method provided by the present invention, as shown below. Figure 6 As shown, step 130 specifically includes:

[0122] Step 131: Based on the feature point group, determine the intrinsic and extrinsic parameters of the image acquisition device and its pose at each viewpoint, and reconstruct the point cloud of the feature points in the images at least two viewpoints to obtain the initial point cloud data of the object to be detected.

[0123] Step 132: Based on the 3D points in the initial point cloud data and the pose of the image acquisition device at the reference viewpoint, the initial point cloud data is expanded to obtain the point cloud data of the object to be detected.

[0124] Specifically, by obtaining the feature point groups after matching the feature points in the images from various viewpoints, point cloud reconstruction of the object to be detected can be performed to obtain the point cloud data of the object to be detected.

[0125] The PnP (Perspective-n-Point) algorithm can be used to determine the intrinsic and extrinsic parameters of the image acquisition device (camera intrinsic parameter matrix K, extrinsic parameters including rotation matrix R and translation matrix T), as well as the pose estimation of the image acquisition device under various viewpoints, and reconstruct the 3D space of the feature points of the object to be detected to obtain the initial point cloud data. It is understandable that the initial point cloud data is relatively sparse.

[0126] Based on this, the initial point cloud data can be further expanded to obtain the point cloud data of the object to be detected, which is denser than the initial point cloud data.

[0127] Select one of the images of the object to be detected from at least two perspectives as a reference image. The reference perspective is the shooting perspective of the reference image. The pose of the image acquisition device under the reference perspective can be determined by establishing the camera coordinate system of the reference image.

[0128] Then, seed points are acquired by selecting 3D points from the initial point cloud data that can be projected onto the reference image. These 3D points are projected onto the reference image, with the distance from the 3D point to the origin of the camera coordinate system in the reference image serving as the initial depth of the projected pixel, and the direction from the 3D point to the origin of the camera coordinate system serving as the initial normal vector. The depth and normal vector information of the remaining pixels are then recovered. These pixels are used as seeds to optimize and solve for the depth and normal vector information of other pixels in the reference image, generating a depth map.

[0129] When expanding, the region growing method can be used, following the three selection principles:

[0130] 1. Depth estimation starts from the initial sparse feature points;

[0131] 2. Perform nonlinear depth optimization on each seed point;

[0132] 3. After each optimization, determine the following two situations and add the neighboring pixels to the queue: 1) The neighborhood has no depth value; 2) The confidence value of the current pixel is higher than that of the neighboring pixels by a certain range.

[0133] Finally, the point cloud data of the object to be detected is obtained.

[0134] Based on any of the above embodiments, assuming the object to be detected is a dead pig, Figure 7 This is a second schematic flowchart of the weight detection method provided by the present invention, as shown below. Figure 7 As shown, the method includes:

[0135] Step 710, Sample Image Acquisition: Acquire images of the dead pig from at least two perspectives using a mobile device.

[0136] Data can be collected using commercially available mobile phones. A data collection application should be prepared in advance, displaying a simulated outline of a pig against a light-colored background, along with multiple marker areas (four-leg area, tail area, area between the four legs and tail, head area, and body area). Position the dead pig in a side-lying position. During data collection, move the phone so that the pig is completely within the pre-drawn outline in the application, and the markers are completely within the application's marker frame, then take a picture. Move the marker position to the next pre-set marker area and repeat the above collection process. This allows for the collection of images of the pig from different perspectives. Simultaneously, record the pig's actual weight. Data should be collected under different lighting conditions and at different times of death (different times of death affect the pig's water loss and further influence its surface morphology).

[0137] Step 720, sample image preprocessing, may include image size unification, distortion correction, and pig contour extraction, etc.

[0138] Since different mobile phones have different default resolutions, exposures, and field of view parameters, data preprocessing is necessary to make the model easier to train. First, all images are resized to fixed width and height values, for example: 480 pixels wide and 640 pixels high.

[0139] Next, distortion correction is performed to restore the objects in the image to their true proportions.

[0140] Pig contour extraction: Since the posture, shape, and color of the dead pig are known, the color thresholding method is first used to coarsely extract the contour of the dead pig and distinguish the background. Then, a simple fitting shape primitive method is used to first extract all closed contours in the image, and then the fitting method is used to obtain objects that are potential pigs. If there are multiple targets, the template matching method is used to remove interference.

[0141] Step 730: Based on sample images of the dead pig from at least two viewpoints, and the matching feature points annotated in the sample images, train a feature point extraction and matching model. The feature point extraction and matching model may specifically include a feature point detector, an orientation estimation model, and a feature extraction model.

[0142] During training, the feature extraction model is trained first, followed by the orientation estimation model, and finally the feature point detector. Multiple images (2-4 images) with labeled pairs of matching points are used for training. The training methods for the feature extraction model and the orientation estimation model are described in the previous embodiment and will not be repeated here.

[0143] The training process of the feature point detector is as follows: A Gaussian kernel function is used to generate a heatmap for the matched feature points. This means that each class of coordinates is represented by a probability map. Each pixel in the image is assigned a probability, indicating the probability that the point belongs to the corresponding class of feature points. Naturally, the closer a pixel is to a feature point, the closer its probability is to 1, and the farther away a pixel is, the closer its probability is to 0. This can be simulated using appropriate functions, such as the 2D Gaussian function. If the distance of the same pixel to different feature points is different, meaning the probability of that pixel at that location is different relative to different feature points, then a Max or Average value can be used. Finally, a focal loss is used to constrain the training.

[0144] Step 740: Obtain images of the dead pig to be detected from at least two perspectives, and use a trained feature extraction and matching model to perform feature point matching on the images to obtain feature point groups.

[0145] Step 750: Based on the feature point set, determine the intrinsic and extrinsic parameters of the camera and perform camera pose estimation at each viewpoint.

[0146] Step 760: Based on the camera poses of each viewpoint, unify the viewpoints to the same coordinate system, first convert them into depth maps, and then use the region growing and dilatation method to obtain the dead pig point cloud data after point cloud reconstruction.

[0147] Step 770: Based on the body length of the dead pig, the first template point cloud data that matches the body length is selected from the template point cloud data. The weight of the first template point cloud data with the smallest registration loss with the point cloud data is determined as the weight of the dead pig to be detected.

[0148] The weight detection device provided by the present invention is described below. The weight detection device described below can be referred to in correspondence with the weight detection method described above.

[0149] Based on any of the above embodiments Figure 8 This is a schematic diagram of the weight detection device provided by the present invention, as shown below. Figure 8 As shown, the device includes an image acquisition unit 810, a feature point matching unit 820, a point cloud reconstruction unit 830, and a point cloud matching unit 840, wherein...

[0150] The image acquisition unit 810 is used to acquire images of the object to be detected from at least two viewpoints;

[0151] The feature point matching unit 820 is used to perform feature point matching on the feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints.

[0152] The point cloud reconstruction unit 830 is used to reconstruct the point cloud of the object to be detected based on the feature point group, so as to obtain the point cloud data of the object to be detected.

[0153] The point cloud matching unit 840 is used to match the point cloud data with template point cloud data, and determine the weight of the template point cloud data that matches the point cloud data as the weight of the object to be detected.

[0154] The weight detection device provided in this invention extracts feature points from images at least two different viewpoints and matches these feature points between the images to obtain feature point groups. It then reconstructs a point cloud from these feature point groups to obtain point cloud data. The reconstructed point cloud data is matched with template point cloud data in a template database; the weight corresponding to the successfully matched template point cloud data is then taken as the weight of the object to be detected. This device enables contactless, automated, and intelligent weight detection of the object, avoiding the problems associated with manual weighing in existing technologies. It not only improves the efficiency of weight detection and reduces labor costs but also enhances accuracy and reliability.

[0155] Based on any of the above embodiments, the feature point matching unit is further configured to:

[0156] Based on the shooting angle of the images from each viewpoint, the feature point patches in the images from each viewpoint are spatially transformed, and the features of the spatially transformed feature point patches are extracted to obtain the feature point description information of each viewpoint in the same space.

[0157] Based on the feature point description information of the at least two viewpoints, feature point matching is performed on the feature points in the images under the at least two viewpoints to obtain feature point groups.

[0158] Based on any of the above embodiments, a shooting angle acquisition unit is further included, used for:

[0159] Based on the feature point patches where the feature points are located in the images from each viewpoint, the orientation of the images from each viewpoint is estimated to obtain the shooting angle from each viewpoint.

[0160] Based on any of the above embodiments, the shooting angle acquisition unit is further used for:

[0161] Based on the orientation estimation model, the orientation of the images under each viewpoint is estimated by applying the feature point patch where the feature points are located in the images under each viewpoint, and the shooting angle under each viewpoint is obtained.

[0162] The orientation estimation model is trained based on the patch of sample 3D points under different viewpoints, with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point under different viewpoints.

[0163] Based on any of the above embodiments, the feature point matching unit is further configured to:

[0164] Based on the feature extraction model, the feature point map patch after spatial transformation is applied to extract features from the feature point map patch to obtain the feature point description information of the feature point map patch.

[0165] The feature point description information of the feature point patch is mapped to the image of each viewpoint to which the feature point patch belongs, so as to obtain the feature point description information of each viewpoint in the same space;

[0166] The feature extraction model is trained based on the similarity between feature point patches containing matching feature points in sample images from at least two viewpoints, and / or the similarity between feature point patches containing non-matching feature points.

[0167] Based on any of the above embodiments, the point cloud matching unit is further used for:

[0168] Based on the point cloud data, the body length of the object to be detected is determined;

[0169] Based on the body length of the object to be detected, first template point cloud data matching the body length is obtained by filtering from the template point cloud data;

[0170] The point cloud data is matched with the first template point cloud data.

[0171] Based on any of the above embodiments, the point cloud reconstruction unit is further used for:

[0172] Based on the feature point group, the intrinsic and extrinsic parameters of the image acquisition device and its pose at each viewpoint are determined, and point cloud reconstruction is performed on the feature points in the images at least two viewpoints to obtain the initial point cloud data of the object to be detected.

[0173] Based on the three-dimensional points in the initial point cloud data and the pose of the image acquisition device at the reference viewpoint, the initial point cloud data is expanded to obtain the point cloud data of the object to be detected.

[0174] Based on any of the above embodiments Figure 9 This is a schematic diagram of the weight detection system provided by the present invention, as shown below. Figure 9 As shown, the weight detection system includes an image acquisition device 910 and a weight detection device 920, wherein,

[0175] Image acquisition device 910 is used to acquire images of the object to be detected from at least two perspectives and send the images from at least two perspectives to the weight detection device;

[0176] The weight detection device 920 is used to perform feature point matching on feature points in images from at least two viewpoints to obtain a feature point group. Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain point cloud data of the object to be detected. The point cloud data is then matched with template point cloud data. The weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected. The feature point group includes feature points from at least two viewpoints.

[0177] Specifically, the weight detection system provided in this embodiment of the invention can automatically and intelligently detect the weight of the object to be detected. The image acquisition device is used to acquire images of the object to be detected from at least two perspectives; for example, it can be a smartphone with a camera function.

[0178] The weight detection device is used to perform weight detection based on images acquired from at least two viewpoints. The specific detection process has been described in detail in the foregoing embodiments and will not be repeated here.

[0179] The weight detection device and the image acquisition device can be integrated into the same electronic device, such as a smartphone. The smartphone takes images of the object to be detected from at least two angles and then sends the images to the weight detection device for weight detection.

[0180] The weight detection equipment can also be set up in the cloud. The image acquisition equipment sends the acquired images to the weight detection equipment via the network, and the weight detection is completed in the cloud.

[0181] The weight detection system provided in this invention can automatically and intelligently detect the weight of the object to be detected, thereby avoiding the problems caused by the need for manual weighing in the prior art. It not only improves the efficiency of weight detection and reduces labor costs, but also improves accuracy and reliability.

[0182] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10 As shown, the electronic device may include: a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communications interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute a weight detection method, which includes:

[0183] Acquire images of the object to be detected from at least two viewpoints;

[0184] Feature point matching is performed on feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints.

[0185] Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain the point cloud data of the object to be detected;

[0186] The point cloud data is matched with the template point cloud data, and the weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected.

[0187] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0188] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to perform the weight detection method provided by the above methods, the method comprising:

[0189] Acquire images of the object to be detected from at least two viewpoints;

[0190] Feature point matching is performed on feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints.

[0191] Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain the point cloud data of the object to be detected;

[0192] The point cloud data is matched with the template point cloud data, and the weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected.

[0193] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the weight detection method provided by the methods described above, the method comprising:

[0194] Acquire images of the object to be detected from at least two viewpoints;

[0195] Feature point matching is performed on feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints.

[0196] Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain the point cloud data of the object to be detected;

[0197] The point cloud data is matched with the template point cloud data, and the weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected.

[0198] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0199] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A weight detection method, characterized in that, include: Acquire images of the object to be detected from at least two viewpoints; Feature point matching is performed on feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints. Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain the point cloud data of the object to be detected; The point cloud data is matched with the template point cloud data, and the weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected. The template point cloud data that matches the point cloud data is the template point cloud data with the smallest registration loss with the object to be detected. The step of matching the point cloud data with the template point cloud data includes: When the object to be detected is livestock, the body length of the object to be detected is determined based on the point cloud data; Based on the body length of the object to be detected, first template point cloud data matching the body length is obtained by filtering from the template point cloud data; Remove the head data from the point cloud data to obtain point cloud data containing only the body. Calculate the registration loss between the point cloud data containing only the body and the first template point cloud data. Then, select the first template point cloud data with the smallest registration loss to the point cloud data containing only the body as the matching template point cloud data. The step of matching feature points in the images from at least two viewpoints to obtain feature point groups includes: Based on the shooting angle of the images from each viewpoint, the feature point patches in the images from each viewpoint are spatially transformed, and the features of the spatially transformed feature point patches are extracted to obtain the feature point description information of each viewpoint in the same space. Based on the feature point description information of the at least two perspectives, feature point matching is performed on the feature points in the images under the at least two perspectives to obtain feature point groups; The steps for determining the shooting angle include: Based on the orientation estimation model, the orientation of the images under each viewpoint is estimated by applying the feature point patch where the feature points are located in the images under each viewpoint, and the shooting angle under each viewpoint is obtained. The orientation estimation model is trained based on the patch of sample 3D points under different viewpoints, with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point under different viewpoints.

2. The weight detection method according to claim 1, characterized in that, The step of extracting features from the spatially transformed feature point map to obtain feature point description information for each viewpoint in the same space includes: Based on the feature extraction model, the feature point map patch after spatial transformation is applied to extract features from the feature point map patch to obtain the feature point description information of the feature point map patch. The feature point description information of the feature point patch is mapped to the image of each viewpoint to which the feature point patch belongs, so as to obtain the feature point description information of each viewpoint in the same space; The feature extraction model is trained based on the similarity between feature point patches containing matching feature points in sample images from at least two viewpoints, and / or the similarity between feature point patches containing non-matching feature points.

3. The weight detection method according to any one of claims 1 to 2, characterized in that, The step of reconstructing the point cloud of the object to be detected based on the feature point group to obtain the point cloud data of the object to be detected includes: Based on the feature point group, the intrinsic and extrinsic parameters of the image acquisition device and its pose at each viewpoint are determined, and point cloud reconstruction is performed on the feature points in the images at least two viewpoints to obtain the initial point cloud data of the object to be detected. Based on the three-dimensional points in the initial point cloud data and the pose of the image acquisition device at the reference viewpoint, the initial point cloud data is expanded to obtain the point cloud data of the object to be detected.

4. A weight detection device, characterized in that, include: An image acquisition unit is used to acquire images of the object to be detected from at least two viewpoints. A feature point matching unit is used to perform feature point matching on feature points in the images from at least two viewpoints to obtain a feature point group, wherein the feature point group includes feature points from at least two viewpoints. The point cloud reconstruction unit is used to reconstruct the point cloud of the object to be detected based on the feature point group, so as to obtain the point cloud data of the object to be detected. A point cloud matching unit is used to match the point cloud data with template point cloud data, determine the weight of the template point cloud data that matches the point cloud data as the weight of the object to be detected, and the template point cloud data that matches the point cloud data is the template point cloud data with the minimum registration loss between it and the object to be detected. The step of matching the point cloud data with the template point cloud data includes: When the object to be detected is livestock, the body length of the object to be detected is determined based on the point cloud data; Based on the body length of the object to be detected, first template point cloud data matching the body length is obtained by filtering from the template point cloud data; Remove the head data from the point cloud data to obtain point cloud data containing only the body. Calculate the registration loss between the point cloud data containing only the body and the first template point cloud data. Then, select the first template point cloud data with the smallest registration loss to the point cloud data containing only the body as the matching template point cloud data. The feature point matching unit is specifically used for: Based on the shooting angle of the images from each viewpoint, the feature point patches in the images from each viewpoint are spatially transformed, and the features of the spatially transformed feature point patches are extracted to obtain the feature point description information of each viewpoint in the same space. Based on the feature point description information of the at least two perspectives, feature point matching is performed on the feature points in the images under the at least two perspectives to obtain feature point groups; The steps for determining the shooting angle include: Based on the orientation estimation model, the orientation of the images under each viewpoint is estimated by applying the feature point patch where the feature points are located in the images under each viewpoint, and the shooting angle under each viewpoint is obtained. The orientation estimation model is trained based on the patch of sample 3D points under different viewpoints, with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point under different viewpoints.

5. A weight detection system, characterized in that, This includes image acquisition equipment and weight detection equipment; The image acquisition device is used to acquire images of the object to be detected from at least two perspectives, and to send the images from the at least two perspectives to the weight detection device; The weight detection device is used to perform feature point matching on feature points in the images from at least two viewpoints to obtain a feature point group. Based on the feature point group, point cloud reconstruction is performed on the object to be detected to obtain point cloud data of the object to be detected. The point cloud data is then matched with template point cloud data. The weight of the template point cloud data that matches the point cloud data is determined as the weight of the object to be detected. The feature point group includes feature points from at least two viewpoints. The template point cloud data that matches the point cloud data is the template point cloud data with the minimum registration loss between it and the object to be detected. The step of matching the point cloud data with the template point cloud data includes: When the object to be detected is livestock, the body length of the object to be detected is determined based on the point cloud data; Based on the body length of the object to be detected, first template point cloud data matching the body length is obtained by filtering from the template point cloud data; Remove the head data from the point cloud data to obtain point cloud data containing only the body. Calculate the registration loss between the point cloud data containing only the body and the first template point cloud data. Then, select the first template point cloud data with the smallest registration loss to the point cloud data containing only the body as the matching template point cloud data. The step of matching feature points in the images from at least two viewpoints to obtain feature point groups includes: Based on the shooting angle of the images from each viewpoint, the feature point patches in the images from each viewpoint are spatially transformed, and the features of the spatially transformed feature point patches are extracted to obtain the feature point description information of each viewpoint in the same space. Based on the feature point description information of the at least two perspectives, feature point matching is performed on the feature points in the images under the at least two perspectives to obtain feature point groups; The steps for determining the shooting angle include: Based on the orientation estimation model, the orientation of the images under each viewpoint is estimated by applying the feature point patch where the feature points are located in the images under each viewpoint, and the shooting angle under each viewpoint is obtained. The orientation estimation model is trained based on the patch of sample 3D points under different viewpoints, with the goal of minimizing the difference in feature point description information corresponding to the same sample 3D point under different viewpoints.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the weight detection method as described in any one of claims 1 to 3.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the weight detection method as described in any one of claims 1 to 3.

Images