Loan pledge monitoring method and device

Abstract

The invention discloses a method and a device for monitoring a loan pledge. The method comprises the following steps: acquiring a binocular picture containing the loan pledge and converting the binocular picture into a discrete point cloud picture; determining the weight of the loan pledge according to the foreground in the discrete point cloud atlas; converting the discrete point cloud picture into an aerial view; detecting the aerial view of the loan prisoner to obtain the number of the loan prisoner; projecting the discrete point cloud image to the two-dimensional image, obtaining two-dimensional image coordinates, extracting color features corresponding to the two-dimensional image coordinates, and adding the color features to the discrete point cloud image to obtain a discrete point cloud image carrying colors; performing identity detection on the loan prisoner in the discrete point cloud picture carrying the color to obtain identity information of the loan prisoner; and determining the weight of the loan prisoner, the number of the loan prisoner and the identity information of the loan prisoner as the information of the loan prisoner. Through binocular vision, point cloud data and image feature extraction, loan prisoner features are comprehensively and accurately captured, inaccuracy of a single information source is avoided, and accurate statistics of prisoner information is ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a monitoring method and device for loan collateral. Background Art

[0002] Farmer loans are an important business direction for banks. Loan collateral refers to a type of property used as collateral or pledge to obtain a loan. Usually, loan collateral can be movable or immovable property. For example, houses and cars are common loan collaterals. Farmer loans also involve crops, livestock, etc.

[0003] At present, cameras are generally used to monitor loan collateral and to count the number of loan collateral. However, when the loan collateral is very crowded, the camera may not be able to accurately capture the information of each loan collateral due to viewing angle limitations, overlapping objects or occlusions, which may lead to inaccurate information on the loan collateral.

[0004] Therefore, how to accurately count the information of loan collateral has become an urgent problem to be solved in this field. Summary of the invention

[0005] The present application provides a monitoring method and device for loan collateral, the purpose of which is to accurately count the information of loan collateral.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A method for monitoring loan collateral, comprising:

[0008] Acquire and convert a binocular image containing the loan collateral into a discrete point cloud image;

[0009] Determine the weight of the loan collateral according to the foreground in the discrete point cloud image; wherein the foreground includes the loan collateral;

[0010] Performing dimensionality reduction and pooling processing on the discrete point cloud image to obtain a bird's-eye view of the loan collateral;

[0011] Detecting the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral;

[0012] Projecting each point cloud data in the discrete point cloud image onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data;

[0013] For each two-dimensional image coordinate corresponding to the point cloud data, extract the image feature corresponding to the two-dimensional image coordinate to obtain the color feature of the image;

[0014] Adding the color features of the image to the discrete point cloud image to obtain a discrete point cloud image carrying color;

[0015] Performing identity detection on the loan collateral in the color-carrying discrete point cloud image to obtain identity information of the loan collateral;

[0016] The weight of the loan collateral, the number of the loan collateral and the identity information of the loan collateral are determined as the information of the loan collateral.

[0017] Optionally, the acquiring and converting the binocular image containing the loan collateral into a discrete point cloud image includes:

[0018] Obtain binocular images of loan collateral;

[0019] Using a backbone network to extract features from the binocular image to obtain feature information;

[0020] Performing face detection on the feature information using a detection algorithm to obtain a left face area and a right face area; the left face area indicates a face area of ​​a first picture in the binocular picture; the right face area indicates a face area of ​​a second picture in the binocular picture, and the first picture and the second picture have different viewing angles;

[0021] Calculating a global disparity based on the left face area and the right face area;

[0022] Based on the depth value calculation formula and the global disparity, the depth value of the pixel point is calculated;

[0023] A depth map is constructed based on the depth values ​​of the pixel points, and the depth map is converted into a discrete point cloud map.

[0024] Optionally, determining the weight of the loan collateral according to the foreground in the discrete point cloud image includes:

[0025] The background in the discrete point cloud image is deleted using the first segmentation algorithm to obtain a foreground image; wherein the background does not include the loan collateral;

[0026] The foreground image is completed using a completion algorithm to obtain a completed point cloud image;

[0027] Using a second segmentation algorithm to extract features from the completed point cloud image to obtain high-dimensional features;

[0028] Performing dimensionality reduction processing on the high-dimensional features by using a dimensionality reduction algorithm to obtain dimensionality reduction features;

[0029] The dimension reduction features are input into the regression model to obtain the weight of the loan collateral.

[0030] Optionally, detecting the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral includes:

[0031] Using a feature extraction network to extract features from the bird's-eye view of the loan collateral to obtain image features;

[0032] Detecting the image features using a 3D detection head to obtain at least one detection frame;

[0033] The number of detection boxes is determined as the number of loan collateral.

[0034] Optionally, performing identity detection on the loan collateral in the color-carrying discrete point cloud image to obtain identity information of the loan collateral includes:

[0035] Extracting features from the discrete point cloud image carrying colors to obtain features of the loan collateral;

[0036] Calculating the similarity between the characteristics of the loan collateral and the preset characteristics to obtain a similarity value;

[0037] If the similarity value exceeds the preset threshold, the identity information of the loan collateral is determined as the identity information corresponding to the preset feature.

[0038] A monitoring device for loan collateral, comprising:

[0039] An acquisition unit, used for acquiring and converting a binocular image containing the loan collateral into a discrete point cloud image;

[0040] A first determining unit is used to determine the weight of the loan collateral according to the foreground in the discrete point cloud image; wherein the foreground includes the loan collateral;

[0041] A dimensionality reduction unit, used for performing dimensionality reduction pooling processing on the discrete point cloud image to obtain a bird's-eye view of the loan collateral;

[0042] A detection unit, used to detect the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral;

[0043] A projection unit, used for projecting each point cloud data in the discrete point cloud image onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data;

[0044] An extraction unit, configured to extract, for each two-dimensional image coordinate corresponding to the point cloud data, an image feature corresponding to the two-dimensional image coordinate, to obtain a color feature of the image;

[0045] An adding unit, used for adding the color feature of the image to the discrete point cloud image to obtain a discrete point cloud image carrying color;

[0046] An identity detection unit, used to perform identity detection on the loan collateral in the color-carrying discrete point cloud image to obtain identity information of the loan collateral;

[0047] The second determining unit is used to determine the weight of the loan collateral, the number of the loan collateral and the identity information of the loan collateral as the loan collateral information.

[0048] Optionally, the acquiring unit is specifically used for:

[0049] Obtain binocular images of loan collateral;

[0050] Using a backbone network to extract features from the binocular image to obtain feature information;

[0051] Performing face detection on the feature information using a detection algorithm to obtain a left face area and a right face area; the left face area indicates a face area of ​​a first picture in the binocular picture; the right face area indicates a face area of ​​a second picture in the binocular picture, and the first picture and the second picture have different viewing angles;

[0052] Calculating a global disparity based on the left face area and the right face area;

[0053] Based on the depth value calculation formula and the global disparity, the depth value of the pixel point is calculated;

[0054] A depth map is constructed based on the depth values ​​of the pixel points, and the depth map is converted into a discrete point cloud map.

[0055] Optionally, the first determining unit is specifically configured to:

[0056] The background in the discrete point cloud image is deleted using the first segmentation algorithm to obtain a foreground image; wherein the background does not include the loan collateral;

[0057] The foreground image is completed using a completion algorithm to obtain a completed point cloud image;

[0058] Using a second segmentation algorithm to extract features from the completed point cloud image to obtain high-dimensional features;

[0059] Performing dimensionality reduction processing on the high-dimensional features by using a dimensionality reduction algorithm to obtain dimensionality reduction features;

[0060] The dimension reduction features are input into the regression model to obtain the weight of the loan collateral.

[0061] Optionally, the detection unit is specifically used for:

[0062] Using a feature extraction network to extract features from the bird's-eye view of the loan collateral to obtain image features;

[0063] Detecting the image features using a 3D detection head to obtain at least one detection frame;

[0064] The number of detection boxes is determined as the number of loan collateral.

[0065] Optionally, the identity detection unit is specifically used for:

[0066] Extracting features from the discrete point cloud image carrying colors to obtain features of the loan collateral;

[0067] Calculating the similarity between the characteristics of the loan collateral and the preset characteristics to obtain a similarity value;

[0068] If the similarity value exceeds the preset threshold, the identity information of the loan collateral is determined as the identity information corresponding to the preset feature.

[0069] The technical solution provided by the present application discloses a monitoring method and device for loan collateral, which obtains and converts a binocular image containing loan collateral into a discrete point cloud image; determines the weight of the loan collateral according to the foreground in the discrete point cloud image; performs dimensionality reduction pooling processing on the discrete point cloud image to obtain a bird's-eye view of the loan collateral; detects the loan collateral in the bird's-eye view of the loan collateral to obtain the number of loan collateral; projects each point cloud data in the discrete point cloud image onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data; extracts the image features corresponding to the two-dimensional image coordinates for each point cloud data to obtain the color features of the image; adds the color features of the image to the discrete point cloud image to obtain a discrete point cloud image with color; performs identity detection on the loan collateral in the discrete point cloud image with color to obtain the identity information of the loan collateral; determines the weight of the loan collateral, the number of the loan collateral and the identity information of the loan collateral as the information of the loan collateral. Through binocular vision, point cloud data and image feature extraction, the loan collateral features are fully and accurately captured to avoid the inaccuracy of a single information source and ensure accurate statistics of collateral information. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 A flowchart of a method for monitoring loan collateral provided in an embodiment of the present application;

[0072] Figure 2 A flowchart of a picture conversion method provided in an embodiment of the present application;

[0073] Figure 3A flow chart of a method for determining the weight of a loan collateral provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of a process for determining the weight of a loan collateral provided in an embodiment of the present application;

[0075] Figure 5 A flowchart of a method for determining the amount of loan collateral provided in an embodiment of the present application;

[0076] Figure 6 A schematic diagram of a process for determining the quantity of loan collateral provided in an embodiment of the present application;

[0077] Figure 7 A flowchart of a method for determining identity information of a loan collateral provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of a process for determining identity information of a loan collateral provided in an embodiment of the present application;

[0079] Fig. 9 A schematic diagram of the architecture of a loan collateral monitoring device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0081] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0082] like Figure 1 FIG. 1 is a flowchart of a method for monitoring loan collateral provided in an embodiment of the present application, comprising the following steps:

[0083] S101: Acquire and convert a binocular image containing the loan collateral into a discrete point cloud image.

[0084] Among them, the binocular image is taken by a binocular camera. The binocular image includes a left image and a right image. There will be parallax between the left image and the right image during the imaging process, that is, the horizontal coordinates of the same point projected in the left image and the right image are different.

[0085] It should be noted that the binocular image may contain only one loan collateral or a group of loan collateral. When detecting the number of loan collaterals, the binocular image contains a group of loan collaterals, and when detecting the weight and identity of the loan collaterals, the binocular image contains only one loan collateral.

[0086] Optionally, the discrete point cloud image is a picture of a three-dimensional space.

[0087] It can be understood that the binocular image containing the loan collateral is converted into a discrete point cloud image. Specifically, the binocular image containing the loan collateral is subjected to feature extraction through a binocular depth estimation algorithm to obtain a depth map, and the depth map is converted into a discrete point cloud image.

[0088] It should be noted that loan collateral includes but is not limited to livestock (for example, cattle and sheep).

[0089] Optionally, in another embodiment of the present application, the specific implementation of step S101 is as follows: Figure 2 As shown, the following steps are included:

[0090] S201: Obtain a binocular image of the loan collateral.

[0091] S202: Using the backbone network to extract features from the binocular image to obtain feature information.

[0092] The feature information includes but is not limited to: shape, texture, color, edge, corner point, and texture.

[0093] Optionally, the backbone network includes but is not limited to the Resnet series. In order to improve feature extraction efficiency and performance, Resnet54 is used in this application.

[0094] S203: Performing face detection on the feature information using a detection algorithm to obtain a left face region and a right face region.

[0095] The left face area indicates the face area of ​​the first picture in the binocular picture, and the right face area indicates the face area of ​​the second picture in the binocular picture. The first picture and the second picture have different viewing angles.

[0096] Optionally, the detection algorithm includes but is not limited to a YOLO algorithm, and the YOLO algorithm is used to perform face detection on the feature information to perform 2D detection to obtain a left face area and a right face area, as well as coordinates of the left face area and coordinates of the right face area.

[0097] S204: Calculate the global disparity based on the left face area and the right face area.

[0098] It can be understood that the global disparity is calculated based on the left face area and the right face area. Specifically, the same facial features are first screened out from the left face area and the right face area, and the horizontal coordinates of the facial features of the left face area are marked as the first horizontal coordinate, and the horizontal coordinates of the facial features of the right face area are marked as the second horizontal coordinate; the difference between the first horizontal coordinate and the second horizontal coordinate is calculated to obtain the local disparity; the minimum horizontal coordinate of the left face area is obtained and marked as the first minimum horizontal coordinate, and the minimum horizontal coordinate of the right face area is obtained and marked as the second minimum horizontal coordinate; the difference between the first minimum horizontal coordinate and the second minimum horizontal coordinate is calculated, and the sum of the difference and the local disparity is calculated to obtain the global disparity.

[0099] S205: Calculate the depth value of the pixel based on the depth value calculation formula and the global disparity.

[0100] The specific expression of the depth value calculation formula is shown in formula (1).

[0101] Z = f*b / disp(global)

[0102] In formula (1), f is the focal length, b is the baseline (i.e., the distance between the two focal points), Z is the depth value of the pixel, and disp(global) is the global disparity.

[0103] S206: construct a depth map based on the depth values ​​of the pixel points, and convert the depth map into a discrete point cloud map.

[0104] It can be understood that since the coordinates of a point in the depth map are p(u,v,d), where u and v represent the coordinates in the image coordinate system in pixels, and d represents the depth in meters. The coordinate p is in the view cone space, which reflects the process of projecting the real 3D space into the image space and can be understood as a distorted 3D space. In order to remove this distortion, the depth map of the view cone space is converted into a discrete point cloud map of the 3D space.

[0105] It should be noted that the depth map can be converted into a discrete point cloud map through a projection formula, wherein the specific expression of the projection formula is shown in formula (2).

[0106] ZP=K(R*Pw+t)(2)

[0107] In formula (2), Z represents the z coordinate data in the 3D space, which is equal to the depth d here, K represents the camera intrinsic parameter, R represents the rotation matrix, t represents the translation matrix, which belongs to the camera extrinsic parameter, and Pw represents the coordinate in the 3D coordinate system.

[0108] It should be emphasized that the coordinates of any point in the discrete point cloud can be expressed as Pw(x, y, z), where x, y, and z represent the width, height, and length coordinates in the 3D space coordinate system in meters.

[0109] S102: Determine the weight of the loan collateral according to the foreground in the discrete point cloud image.

[0110] Among them, the outlook includes loan collateral.

[0111] It can be understood that the weight of the loan collateral is determined according to the foreground in the discrete point cloud image. Specifically, feature extraction is performed on the foreground in the discrete point cloud image to obtain features; regression fitting is performed on the features to obtain the weight of the loan collateral.

[0112] Optionally, in another embodiment of the present application, the specific implementation of step S102 is as follows: Figure 3 As shown, the following steps are included:

[0113] S301: Utilize the first segmentation algorithm to delete the background in the discrete point cloud image to obtain a foreground image.

[0114] Among them, the background does not include loan collateral.

[0115] Optionally, the first segmentation algorithm includes but is not limited to: Pointnet.

[0116] S302: Use a completion algorithm to complete the foreground image to obtain a completed point cloud image.

[0117] It is understandable that the foreground image obtained by the first segmentation algorithm is equivalent to a 3D reconstruction of the foreground image. However, in the processing process, a part of the complete 3D model of the foreground target will inevitably be occluded, resulting in partial missing of the reconstructed 3D loan collateral. In order to reduce the impact of this situation on weight estimation, a completion algorithm (such as a point cloud completion algorithm) is used to complete the foreground image to obtain a completed point cloud image.

[0118] It should be noted that the point cloud algorithm will be pre-trained on a large number of point cloud data sets, and then refined on specific 3D loan collateral to obtain a completed point cloud map.

[0119] S303: Using the second segmentation algorithm to extract features from the completed point cloud image to obtain high-dimensional features.

[0120] Among them, high-dimensional features include edges and contours, texture features (i.e., texture patterns in images, such as the density or looseness of wool, the fur texture of sheep), and color features.

[0121] Optionally, the second segmentation algorithm includes but is not limited to: an improved Pointnet.

[0122] S304: Using a dimensionality reduction algorithm to reduce the dimensionality of the high-dimensional features to obtain reduced-dimensional features.

[0123] It is understandable that high-dimensional features are compressed into lower-dimensional vectors through dimensionality reduction algorithms (such as PCA or t-SNE), reducing computational complexity and storage requirements while retaining as much key information as possible.

[0124] S305: Input the dimension reduction features into the regression model to obtain the weight of the loan collateral.

[0125] Optionally, the regression model includes but is not limited to: a neural network regression model.

[0126] It should be noted that the regression model is a pre-trained model, which is based on the sample dimensionality reduction features as input and the weight corresponding to the dimensionality reduction features manually annotated on the sample dimensionality reduction features as the training target, and is pre-trained.

[0127] For better Figure 3 The following is an explanation of the content shown in the figure. The following is an explanation of the specific application scenario. Figure 4 , Figure 4 The figure shows a binocular picture of a sheep taken with a binocular camera. The binocular depth estimation algorithm is used to extract features from the binocular picture to obtain a depth map. The depth map is converted into a 3D discrete point cloud map. The 3D semantic segmentation algorithm is used to delete the background in the discrete point cloud map to obtain a foreground map. The point cloud completion algorithm is used to complete the foreground map to obtain a completed point cloud map (i.e., a 3D reconstructed point cloud map). The completed point cloud map is upgraded in dimensionality for feature extraction, and then reduced in dimensionality for regression fitting. Finally, the weight of the loan collateral is output.

[0128] It can be seen that by using binocular images, the loan collateral is reconstructed in three dimensions to obtain the real volume information, and further regressed to obtain the weight, which fundamentally solves the problem of "near is big and far is small", that is, the weight of the loan collateral near is higher than that of the loan collateral far away.

[0129] S103: Perform dimensionality reduction and pooling processing on the discrete point cloud image to obtain a bird's-eye view of the loan collateral.

[0130] It is understandable that since the point cloud data contained in the discrete point cloud image is actually four-dimensional data, namely (n, x, y, z), n is the dimension, the n dimension needs to be upgraded in the subsequent 3D detection process, and the candidate box needs to be searched in the 3D space during the detection process, which is too computationally expensive. On the other hand, in the actual detection process, more attention is paid to the position coordinates (length and width) of the target on the bird's-eye view, and the information contained in the height dimension has little effect on counting. Therefore, this paper pools the point cloud data along the height dimension, reduces the dimension of (n, x, y, z) to (n, x, z), and obtains a bird's-eye view of the loan collateral.

[0131] It should be emphasized that the bird's-eye view can better distinguish targets and alleviate the impact of occlusion and truncation.

[0132] S104: Detect the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral.

[0133] The loan collateral in the bird's-eye view of the loan collateral is detected. Specifically, feature extraction is performed on the bird's-eye view of the loan collateral to obtain image features, and the image features are detected to obtain the number of loan collateral.

[0134] Optionally, in another embodiment of the present application, the specific implementation of step S104 is as follows: Figure 5 As shown, the following steps are included:

[0135] S501: Extract features from the bird's-eye view of the loan collateral using a feature extraction network to obtain image features.

[0136] The image features include but are not limited to: edge, shape, and color.

[0137] It should be noted that the feature extraction network is used to perform dimensionality-upgraded feature extraction on the bird's-eye view of the loan collateral, so as to obtain more comprehensive image features.

[0138] S502: Detect image features using a 3D detection head to obtain at least one detection frame.

[0139] Among them, a detection box contains a loan collateral.

[0140] It can be understood that the detection box can be described as (x, y, w, l, θ) where x and y represent the coordinates of the center point of the detection box, w and l represent the size of the detection box, and θ represents the deviation angle of the box.

[0141] S503: Determine the number of detection frames as the number of loan collaterals.

[0142] It can be understood that, since one detection frame contains one loan collateral, the number of loan collaterals can be determined by counting the number of detection frames, and therefore, the number of detection frames is determined as the number of loan collaterals.

[0143] For better Figure 5 The following is an explanation of the content shown in the figure. The following is an explanation of the specific application scenario. Figure 6 , Figure 6 The figure shows a binocular picture of sheep taken with a binocular camera. A binocular depth estimation algorithm is used to extract features from the binocular picture to obtain a depth map. The depth map is converted into a 3D discrete point cloud map, and the discrete point cloud map is subjected to dimensionality reduction and pooling processing to obtain a bird's-eye view of the loan collateral. 3D detection is performed on the bird's-eye view map to obtain a detection frame, and the number of detection frames is determined as the number of loan collateral.

[0144] from Figure 6 It can be seen that with the help of the binocular camera as a sensor, the flock of sheep is projected into the bird's-eye view (bev) for detection, which effectively reduces the impact of occlusion and truncation.

[0145] S105: Project each point cloud data in the discrete point cloud image onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data.

[0146] Optionally, each point cloud data (ie, Pw(x, y, z)) in the discrete point cloud image can be projected onto a two-dimensional image using formula (2) to obtain the two-dimensional image coordinates (ie, p(u, v)) corresponding to each point cloud data.

[0147] S106: For each two-dimensional image coordinate corresponding to each point cloud data, extract the image features corresponding to the two-dimensional image coordinates to obtain the color features of the image.

[0148] It can be understood that for each two-dimensional image coordinate corresponding to each point cloud data, the image features corresponding to the two-dimensional image coordinates are extracted, that is, the color features corresponding to the two-dimensional image coordinates are collected, and then the color features of each two-dimensional coordinate are obtained.

[0149] S107: Add the color features of the image to the discrete point cloud image to obtain a discrete point cloud image carrying colors.

[0150] It can be understood that for the color feature of each two-dimensional coordinate, the color feature of the two-dimensional coordinate is added to the point cloud data in the discrete point cloud map corresponding to the two-dimensional coordinate, thereby obtaining a discrete point cloud map carrying color (ie (n, x, y, z, c)).

[0151] S108: Perform identity detection on the loan collateral in the discrete point cloud image with colors to obtain identity information of the loan collateral.

[0152] The identity information of the loan collateral indicates the real identity of the loan collateral (eg, sheep, cattle).

[0153] It should be noted that, since the discrete point cloud image with colors incorporates the shape features and texture features of the loan collateral, it is easier to detect the identity information of the loan collateral in the discrete point cloud image.

[0154] Optionally, in another embodiment of the present application, the specific implementation of step S108 is as follows: Figure 7 As shown, the following steps are included:

[0155] S701: Extract features from the discrete point cloud image with colors to obtain features of the loan collateral.

[0156] Among them, the characteristics of the loan collateral include but are not limited to: geometric characteristics and color characteristics.

[0157] S702: Calculate the similarity between the characteristics of the loan collateral and the preset characteristics to obtain a similarity value.

[0158] It is understandable that the cosine similarity between the characteristics of the loan collateral and the preset characteristics can be calculated, or the similarity between the characteristics of the loan collateral and the preset characteristics can be calculated by Euclidean distance.

[0159] For example, if the loan collateral is a cow, the similarity between the features of the loan collateral and the preset features of the cow is calculated to obtain a similarity value.

[0160] S703: If the similarity value exceeds a preset threshold, the identity information of the loan collateral is determined as the identity information corresponding to the preset feature.

[0161] For example, the preset feature is a feature of a cow, and the similarity value exceeds a preset threshold value, and the identity information of the loan collateral is determined to be the identity information of the loan collateral corresponding to the preset feature, that is, the identity information of the loan collateral is determined to be a cow.

[0162] It is understandable that if the similarity value does not exceed the preset threshold, the identity information of the loan collateral is inconsistent with the identity information corresponding to the feature.

[0163] It should be emphasized that the preset threshold can be set according to actual conditions and is not specifically limited here.

[0164] For better Figure 7 The following is an explanation of the content shown in the figure. The following is an explanation of the specific application scenario. Figure 8 , Figure 8The figure shows a binocular picture of sheep taken with a binocular camera. The binocular depth estimation algorithm is used to extract features of the binocular picture to obtain a depth map. The depth map is converted into a 3D discrete point cloud map, and each point cloud data in the discrete point cloud map is projected onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data. For the two-dimensional image coordinates corresponding to each point cloud data, the image features corresponding to the two-dimensional image coordinates are extracted to obtain the color features of the image. The color features of the image are added to the discrete point cloud map to obtain a discrete point cloud map with color. Identity detection is performed on the loan collateral in the discrete point cloud map with color to obtain the identity information of the loan collateral.

[0165] from Figure 8 It can be seen that the binocular image restores the shape information and combines the original texture information to obtain more discriminative three-dimensional features, thus improving the recognition accuracy and avoiding being deceived.

[0166] S109: Determine the weight of the loan collateral, the number of the loan collateral, and the identity information of the loan collateral as the information of the loan collateral.

[0167] In summary, binocular vision, point cloud data, and image feature extraction can comprehensively and accurately capture the multi-dimensional features of loan collateral, such as geometric shape, color, and quantity. Each step enhances the ability to identify collateral and avoids the inaccuracy that may be caused by a single information source. Therefore, the information of loan collateral can be accurately counted.

[0168] like Fig. 9 As shown, it is a schematic diagram of the architecture of a monitoring device for loan collateral provided in an embodiment of the present application, and the monitoring device includes: an acquisition unit 100, a first determination unit 200, a dimensionality reduction unit 300, a detection unit 400, a projection unit 500, an extraction unit 600, an adding unit 700, an identity detection unit 800 and a second determination unit 900.

[0169] The acquisition unit 100 is used to acquire and convert a binocular image containing the loan collateral into a discrete point cloud image.

[0170] The acquisition unit 100 is specifically used to: acquire a binocular image of the loan collateral; perform feature extraction on the binocular image using a backbone network to obtain feature information; perform face detection on the feature information using a detection algorithm to obtain a left face area and a right face area; the left face area indicates a face area of ​​a first image in the binocular image; the right face area indicates a face area of ​​a second image in the binocular image, and the first image and the second image have different perspectives; calculate a global disparity based on the left face area and the right face area; calculate a depth value of a pixel point based on a depth value calculation formula and the global disparity; construct a depth map based on the depth value of the pixel point, and convert the depth map into a discrete point cloud map.

[0171] The first determination unit 200 is used to determine the weight of the loan collateral according to the foreground in the discrete point cloud image; wherein the foreground includes the loan collateral.

[0172] The first determination unit 200 is specifically used to: use the first segmentation algorithm to delete the background in the discrete point cloud image to obtain a foreground image; wherein the background does not include the loan collateral; use the completion algorithm to complete the foreground image to obtain a completed point cloud image; use the second segmentation algorithm to extract features of the completed point cloud image to obtain high-dimensional features; use the dimensionality reduction algorithm to reduce the dimensionality of the high-dimensional features to obtain reduced dimensionality features; input the reduced dimensionality features into the regression model to obtain the weight of the loan collateral.

[0173] The dimension reduction unit 300 is used to perform dimension reduction pooling processing on the discrete point cloud image to obtain a bird's-eye view of the loan collateral.

[0174] The detection unit 400 is used to detect the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral.

[0175] The detection unit 400 is specifically used to: extract features from the bird's-eye view of the loan collateral using a feature extraction network to obtain image features; detect the image features using a 3D detection head to obtain at least one detection frame; and determine the number of detection frames as the number of loan collateral.

[0176] The projection unit 500 is used to project each point cloud data in the discrete point cloud image onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data.

[0177] The extraction unit 600 is used to extract the image features corresponding to the two-dimensional image coordinates corresponding to each point cloud data, and obtain the color features of the image.

[0178] The adding unit 700 is used to add the color features of the image to the discrete point cloud image to obtain a discrete point cloud image carrying color.

[0179] The identity detection unit 800 is used to perform identity detection on the loan collateral in the discrete point cloud image carrying colors to obtain the identity information of the loan collateral.

[0180] The identity detection unit 800 is specifically used to: extract features from the discrete point cloud image carrying colors to obtain features of the loan collateral; calculate the similarity between the features of the loan collateral and the preset features to obtain a similarity value; if the similarity value exceeds a preset threshold, determine the identity information of the loan collateral as the identity information corresponding to the preset features.

[0181] The second determining unit 900 is used to determine the weight of the loan collateral, the number of the loan collateral and the identity information of the loan collateral as the information of the loan collateral.

[0182] In summary, binocular vision, point cloud data, and image feature extraction can comprehensively and accurately capture the multi-dimensional features of loan collateral, such as geometric shape, color, and quantity. Each step enhances the ability to identify collateral and avoids the inaccuracy that may be caused by a single information source. Therefore, the information of loan collateral can be accurately counted.

[0183] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0184] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0185] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring loan collateral, characterized in that: include: Acquire and convert a binocular image containing the loan collateral into a discrete point cloud image; Determine the weight of the loan collateral according to the foreground in the discrete point cloud image; wherein the foreground includes the loan collateral; Performing dimensionality reduction and pooling processing on the discrete point cloud image to obtain a bird's-eye view of the loan collateral; Detecting the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral; Projecting each point cloud data in the discrete point cloud image onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data; For each two-dimensional image coordinate corresponding to the point cloud data, extract the image feature corresponding to the two-dimensional image coordinate to obtain the color feature of the image; Adding the color features of the image to the discrete point cloud image to obtain a discrete point cloud image carrying color; Performing identity detection on the loan collateral in the color-carrying discrete point cloud image to obtain identity information of the loan collateral; The weight of the loan collateral, the number of the loan collateral and the identity information of the loan collateral are determined as the information of the loan collateral.

2. The method according to claim 1, characterized in that The step of obtaining and converting a binocular image containing the loan collateral into a discrete point cloud image includes: Obtain binocular images of loan collateral; Using a backbone network to extract features from the binocular image to obtain feature information; Performing face detection on the feature information using a detection algorithm to obtain a left face area and a right face area; the left face area indicates a face area of ​​a first picture in the binocular picture; the right face area indicates a face area of ​​a second picture in the binocular picture, and the first picture and the second picture have different viewing angles; Calculating a global disparity based on the left face area and the right face area; Based on the depth value calculation formula and the global disparity, the depth value of the pixel point is calculated; A depth map is constructed based on the depth values ​​of the pixel points, and the depth map is converted into a discrete point cloud map.

3. The method according to claim 1, characterized in that: The step of determining the weight of the loan collateral according to the foreground in the discrete point cloud image comprises: The background in the discrete point cloud image is deleted using the first segmentation algorithm to obtain a foreground image; wherein the background does not include the loan collateral; The foreground image is completed using a completion algorithm to obtain a completed point cloud image; Using a second segmentation algorithm to extract features from the completed point cloud image to obtain high-dimensional features; Performing dimensionality reduction processing on the high-dimensional features by using a dimensionality reduction algorithm to obtain dimensionality reduction features; The dimension reduction features are input into the regression model to obtain the weight of the loan collateral.

4. The method according to claim 1, characterized in that The detecting the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral includes: Using a feature extraction network to extract features from the bird's-eye view of the loan collateral to obtain image features; Detecting the image features using a 3D detection head to obtain at least one detection frame; The number of detection boxes is determined as the number of loan collateral.

5. The method according to claim 1, characterized in that The performing identity detection on the loan collateral in the color-carrying discrete point cloud image to obtain the identity information of the loan collateral includes: Extracting features from the discrete point cloud image carrying colors to obtain features of the loan collateral; Calculating the similarity between the characteristics of the loan collateral and the preset characteristics to obtain a similarity value; If the similarity value exceeds a preset threshold, the identity information of the loan collateral is determined as the identity information corresponding to the preset feature.

6. A monitoring device for loan collateral, characterized in that: include: An acquisition unit, used for acquiring and converting a binocular image containing the loan collateral into a discrete point cloud image; A first determining unit is used to determine the weight of the loan collateral according to the foreground in the discrete point cloud image; wherein the foreground includes the loan collateral; A dimensionality reduction unit, used for performing dimensionality reduction pooling processing on the discrete point cloud image to obtain a bird's-eye view of the loan collateral; A detection unit, used to detect the loan collateral in the bird's-eye view of the loan collateral to obtain the number of the loan collateral; A projection unit, used for projecting each point cloud data in the discrete point cloud image onto a two-dimensional image to obtain the two-dimensional image coordinates corresponding to each point cloud data; An extraction unit, configured to extract, for each two-dimensional image coordinate corresponding to the point cloud data, an image feature corresponding to the two-dimensional image coordinate, to obtain a color feature of the image; An adding unit, used for adding the color feature of the image to the discrete point cloud image to obtain a discrete point cloud image carrying color; An identity detection unit, used to perform identity detection on the loan collateral in the color-carrying discrete point cloud image to obtain identity information of the loan collateral; The second determining unit is used to determine the weight of the loan collateral, the number of the loan collateral and the identity information of the loan collateral as the loan collateral information.

7. The device according to claim 6, characterized in that The acquisition unit is specifically used for: Obtain binocular images of loan collateral; Using a backbone network to extract features from the binocular image to obtain feature information; Performing face detection on the feature information using a detection algorithm to obtain a left face area and a right face area; the left face area indicates a face area of ​​a first picture in the binocular picture; the right face area indicates a face area of ​​a second picture in the binocular picture, and the first picture and the second picture have different viewing angles; Calculating a global disparity based on the left face area and the right face area; Based on the depth value calculation formula and the global disparity, the depth value of the pixel point is calculated; A depth map is constructed based on the depth values ​​of the pixel points, and the depth map is converted into a discrete point cloud map.

8. The device according to claim 6, characterized in that The first determining unit is specifically configured to: The background in the discrete point cloud image is deleted using the first segmentation algorithm to obtain a foreground image; wherein the background does not include the loan collateral; The foreground image is completed using a completion algorithm to obtain a completed point cloud image; Using a second segmentation algorithm to extract features from the completed point cloud image to obtain high-dimensional features; Performing dimensionality reduction processing on the high-dimensional features by using a dimensionality reduction algorithm to obtain dimensionality reduction features; The dimension reduction features are input into the regression model to obtain the weight of the loan collateral.

9. The device according to claim 6, characterized in that The detection unit is specifically used for: Using a feature extraction network to extract features from the bird's-eye view of the loan collateral to obtain image features; Detecting the image features using a 3D detection head to obtain at least one detection frame; The number of detection boxes is determined as the number of loan collateral.

10. The device according to claim 6, characterized in that The identity detection unit is specifically used for: Extracting features from the discrete point cloud image carrying colors to obtain features of the loan collateral; Calculating the similarity between the characteristics of the loan collateral and the preset characteristics to obtain a similarity value; If the similarity value exceeds a preset threshold, the identity information of the loan collateral is determined as the identity information corresponding to the preset feature.

Images