Object classification method and apparatus, computer device, and storage medium

Abstract

The application relates to an object classification method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining an object region of a target object; dividing the object region of the target object into a plurality of object sub-regions; determining a first correlation feature according to the correlation relationship between points in each object sub-region; determining a second correlation feature according to the correlation relationship between points in each object sub-region and points in adjacent object sub-regions; and determining the category of the target object according to the first correlation feature and the second correlation feature. The application can improve the accuracy of object classification.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an object classification method, apparatus, computer device, and storage medium. Background Technology

[0002] With the rapid development of computer technology, object classification technology, as a branch of computer technology, is being applied more and more widely. Typically, data acquisition devices are used to collect data on target objects, and then the objects are classified based on the collected data. However, this method relies solely on the overall data for object classification, which can lead to low discriminative power and consequently affect the accuracy of object classification. Summary of the Invention

[0003] Therefore, it is necessary to provide an object classification method, apparatus, computer device, computer-readable storage medium, and computer program product to address the aforementioned technical problems and improve the accuracy of object classification.

[0004] Firstly, this application provides an object classification method, including:

[0005] Get the object region of the target object;

[0006] Divide the target object's object region into multiple object sub-regions;

[0007] Based on the relationships between points in each object sub-region, determine the first association feature;

[0008] The second association feature is determined based on the association relationship between points in each object sub-region and points in adjacent object sub-regions;

[0009] The category of the target object is determined based on the first and second association features.

[0010] Secondly, this application provides an object classification apparatus, comprising:

[0011] The acquisition module is used to acquire the object region of the target object;

[0012] The partitioning module is used to divide the object region of a target object into multiple object sub-regions;

[0013] The feature extraction module is used to determine the first association feature based on the relationship between points in each object sub-region; and to determine the second association feature based on the relationship between points in each object sub-region and points in adjacent object sub-regions.

[0014] The classification and recognition module is used to determine the category of the target object based on the first association feature and the second association feature.

[0015] In some embodiments, the object region of the target object includes point cloud blocks, and the object sub-region includes point cloud sub-blocks. The partitioning module is further configured to determine the block center corresponding to the point cloud block of the target object; and to divide the point cloud block into multiple point cloud sub-blocks according to the distance between the block center and each point in the point cloud block.

[0016] In some embodiments, the partitioning module is further configured to determine the maximum distance from the distances between the block center and each point of the point cloud block; determine multiple radii with the block center as the geometric center based on the maximum distance; the multiple radii are less than or equal to the maximum distance; and partition the point cloud block into multiple point cloud sub-blocks based on the multiple radii.

[0017] In some embodiments, the plurality of radii includes a plurality of sphere radii with the block center as the sphere center. The partitioning module is further configured to, for each current sphere radius among the plurality of sphere radii, determine a point cloud sub-block corresponding to the current sphere radius from the point cloud block; wherein, the point cloud sub-block corresponding to the current sphere radius is composed of points located within the region corresponding to the current sphere radius but not within the region corresponding to the reference sphere radius; the reference sphere radius is a sphere radius smaller than the current sphere radius among the plurality of sphere radii.

[0018] In some embodiments, each point in the object sub-region has a corresponding regional feature. The feature extraction module is further configured to determine the association relationship between points in the object sub-region based on the regional features corresponding to each point in the object sub-region, thereby obtaining a first association feature.

[0019] In some embodiments, the region features include first location information; the first location information is the location information of each point in the object sub-region relative to the same reference object. The feature extraction module is further configured to, for each object sub-region, determine second location information between points in the object sub-region based on the first location information corresponding to each point in the object sub-region; and obtain a first association feature based on the second location information.

[0020] In some embodiments, the first location information is the distance of each point in the object sub-region relative to the same reference object; the second location information is the distance between points in the object sub-region. The feature extraction module is further configured to, for each object sub-region, align the coordinates of each point with the reference object according to the first location information corresponding to each point in the object sub-region, to obtain the coordinates of each point relative to the reference object; and determine the distance between points in the object sub-region based on the coordinate differences between points in the object sub-region.

[0021] In some embodiments, the feature extraction module is further configured to, for each object sub-region, determine the association relationship between each point in the object sub-region and each point in the adjacent object sub-region based on the region features corresponding to each point in the object sub-region and the adjacent object sub-region, thereby obtaining a second association feature.

[0022] In some embodiments, the feature extraction module is further configured to, for each object sub-region, determine, based on the first location information corresponding to each point in the object sub-region and each point in the adjacent object sub-region, the third location information between each point in the object sub-region and each point in the adjacent object sub-region; and obtain the second associated feature based on the third location information.

[0023] In some embodiments, the third location information is the distance between each point in the object sub-region and each point in the adjacent object sub-regions. The feature extraction module is further configured to, for each object sub-region, align the coordinates of each point in the object sub-region and each point in the adjacent object sub-regions with the reference object based on the first location information corresponding to each point in the object sub-region and the adjacent object sub-regions, to obtain the coordinates of each point relative to the reference object; and determine the distance between each point in the object sub-region and each point in the adjacent object sub-regions based on the coordinate difference between each point in the object sub-region and each point in the adjacent object sub-regions.

[0024] In some embodiments, the classification and recognition module is further configured to perform weighted fusion of the first associated feature and the second associated feature according to the first weight corresponding to the first associated feature and the second weight corresponding to the second associated feature to obtain fused features; perform dimensionality reduction processing on the fused features to obtain target features; and determine the category of the target object according to the target features.

[0025] In some embodiments, the classification and recognition module is further configured to determine a first weight corresponding to a first associated feature; the first weight is positively correlated with at least one of the number of points or the center distance; the number of points is the number of points in the object sub-region on which the first associated feature is extracted; the center distance is the distance from each point in the object sub-region to the center of the block corresponding to the object region; determine a second weight corresponding to a second associated feature; the second weight is negatively correlated with the first weight; and perform weighted fusion of the first associated feature according to the first weight and the second weight to obtain a fused feature.

[0026] In some embodiments, the category of the target object is obtained by inputting the object region of the target object into an object classification model for classification and recognition. The object classification apparatus of this application embodiment further includes a model training module, used to acquire each sample object region; each sample object region has corresponding object category label information; divide the sample object region into multiple sample object sub-regions using an initial object classification model; determine a first sample association feature based on the association relationship between points in each sample object sub-region; determine a second sample association feature based on the association relationship between points in each sample object sub-region and points in adjacent object sub-regions; predict the object category corresponding to the sample object region based on the first and second sample association features to obtain a category prediction result; and adjust the model parameters of the initial object classification model based on the difference between the category prediction result and the object category label information to perform model training and obtain an object classification model.

[0027] In some embodiments, the category of the target object includes a first category. The object classification apparatus of this application embodiment further includes a category calibration module, configured to send the first category to a smart home device to instruct the smart home device to perform corresponding device control processing according to the first category; receive a second category sent by the smart home device; the second category is obtained by the smart home device after capturing an image of the target object and classifying and recognizing the captured image; and, if the first category and the second category are inconsistent, perform calibration processing on the first category according to the second category.

[0028] Thirdly, this application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method described above.

[0029] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0030] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described above.

[0031] The aforementioned object classification method, apparatus, computer device, computer-readable storage medium, and computer program product acquire the object region of a target object; divide the object region of the target object into multiple object sub-regions; and determine a first association feature based on the correlation between points in each object sub-region. This allows for the extraction of finer-grained feature information from object sub-regions at different scales, enhancing the discriminative power of point features and facilitating subsequent classification tasks. A second association feature is determined based on the correlation between points in each object sub-region and points in adjacent object sub-regions. This strengthens the correlation between points in point object sub-regions at different scales, enabling the extraction of more comprehensive and richer feature information. Furthermore, determining the target object category based on the first and second association features allows for a thorough fusion of these features, simultaneously enhancing the discriminative power and correlation of point features, thereby improving the discriminative power of target object classification and ultimately increasing the accuracy of object classification. Attached Figure Description

[0032] Figure 1 An application environment diagram of an object classification method provided in this application embodiment;

[0033] Figure 2 A flowchart illustrating an object classification method provided in an embodiment of this application;

[0034] Figure 3 A flowchart illustrating another object classification method provided in an embodiment of this application;

[0035] Figure 4 A structural block diagram of an object classification device provided in an embodiment of this application;

[0036] Figure 5 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0038] The object classification method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the data acquisition device 102 communicates with the intelligent device 104 via a network. The data storage system stores the data that the data acquisition device 102 needs to process. The data storage device can be integrated into the data acquisition device 102 or placed in the cloud or on another network server. The data acquisition device 102 acquires the object region of the target object; divides the object region of the target object into multiple object sub-regions; determines a first association feature based on the relationship between points in each object sub-region; determines a second association feature based on the relationship between points in each object sub-region and points in adjacent object sub-regions; and determines the category of the target object based on the first and second association features. Then, the data acquisition device 102 can send the category of the target object to the intelligent device 104 to instruct the intelligent device 104 to perform corresponding device control processing. The data acquisition device 102 can be a point cloud acquisition device, which can be, but is not limited to, at least one of various laser scanners, stereo cameras, or radar devices. Smart device 104 can be a smart home device, which can be, but is not limited to, at least one of various smart lights, smart control panels, smart speakers, smart TVs, smart air conditioners, or smart pet feeders.

[0039] like Figure 2 As shown, this application embodiment provides an object classification method, which is applied to... Figure 1 Taking the data acquisition device 102 as an example, the method includes the following steps:

[0040] Step S202: Obtain the object region of the target object.

[0041] The target object refers to the object that needs to be classified and identified.

[0042] The target area refers to the data collected by data acquisition devices, which includes data from multiple points within the scene where the target object is located, such as an indoor scene.

[0043] Data acquisition equipment refers to instruments used to acquire geometric information about the surface of objects in the environment. In some embodiments, data acquisition equipment can be point cloud acquisition equipment, such as radar equipment. Radar equipment includes millimeter-wave radar equipment, which is a radar system that uses the millimeter-wave frequency band for detection and measurement.

[0044] Specifically, the data acquisition device collects data from the scene containing the target object, directly obtaining the object region containing the target object. Alternatively, after collecting data from the scene containing the target object, the data acquisition device also needs to perform noise reduction processing on the collected data to eliminate these unnecessary noise points and obtain a higher quality object region.

[0045] Step S204: Divide the object region of the target object into multiple object sub-regions.

[0046] Among them, the object sub-region is derived from the object region and is a part of the object region; it can be understood as a local part of the object region.

[0047] Specifically, the data acquisition device can select an appropriate object region division method based on the actual situation of data classification, and divide the object region of the target object into multiple object sub-regions.

[0048] In some embodiments, the object region can be divided using a voxel grid method. That is, the object region can be divided into a regular voxel grid, and points located within the same voxel grid range can be assigned to the same object sub-region to obtain multiple object sub-regions.

[0049] In some embodiments, the object region can be divided based on the position of each point in the object region, so that points that are close in position are assigned to the same object sub-region, resulting in multiple object sub-regions.

[0050] Step S206: Determine the first association feature based on the relationship between points in each object sub-region.

[0051] In this context, the association relationship refers to the relationship between points within each object sub-region in three-dimensional space. It can be understood that the characteristics of each point in three-dimensional space will have certain relationships. For example, each point in an object sub-region has corresponding spatial coordinates in three-dimensional space, and the association relationship between points can be represented by the distance between them, reflecting their proximity in three-dimensional space.

[0052] Specifically, the data acquisition device determines the relationship between points in each object sub-region based on the characteristics of each point in each object sub-region, and then determines the corresponding association features for each object sub-region based on the association relationship, thus obtaining the first association feature.

[0053] Step S208: Determine the second association feature based on the association relationship between points in each object sub-region and points in adjacent object sub-regions.

[0054] The adjacent object sub-region refers to the object sub-region that is adjacent to the corresponding object sub-region.

[0055] Specifically, the data acquisition device determines the association between each point in each object sub-region and each point in adjacent object sub-regions based on the characteristics of each point in each object sub-region, so as to determine the association between each two adjacent object sub-regions based on the association relationship and obtain the second association feature.

[0056] To facilitate understanding, the determination of the first and second association features is illustrated below. Assume the target object's sub-regions include object sub-region A and object sub-region B, where object sub-region A includes points a1, a2, and a3, and object sub-region B includes points b1, b2, and b3. Based on the association relationships between points a1 and a2, a1 and a3, and a2 and a3, the first association feature corresponding to object sub-region A is determined. Based on the association relationships between points b1 and b2, b1 and b3, and b2 and b3, the first association feature corresponding to object sub-region B is determined. Based on the association relationships between points a1 and b1, a1 and b2, a1 and b3, a2 and b1, a2 and b2, and a2 and b3, the second association feature corresponding to object sub-region A and object sub-region B is determined.

[0057] Step S210: Determine the category of the target object based on the first association feature and the second association feature.

[0058] Specifically, the data acquisition device fuses the first and second associated features to determine the category of the target object based on the fused features.

[0059] The object classification method described above obtains the object region of the target object; divides the object region into multiple object sub-regions; and determines the first association feature based on the relationships between points in each object sub-region. This allows for the extraction of finer-grained feature information from object sub-regions at different scales, enhancing the discriminative power of point features and facilitating subsequent classification tasks. The second association feature is determined based on the relationships between points in each object sub-region and points in adjacent object sub-regions. This strengthens the correlation between points in object sub-regions at different scales, enabling the extraction of more comprehensive and richer feature information. Furthermore, determining the target object category based on the first and second association features allows for a thorough fusion of these features, simultaneously enhancing the discriminative power and correlation of point features, thereby improving the discriminative power of target object classification and ultimately increasing the accuracy of object classification.

[0060] In some embodiments, the object region of the target object includes point cloud blocks, and the object sub-region includes point cloud sub-blocks. Step S204 specifically includes, but is not limited to: determining the block center corresponding to the point cloud block of the target object; and dividing the point cloud block into multiple point cloud sub-blocks according to the distance between the block center and each point in the point cloud block.

[0061] Here, the block center refers to the central position of all points in a point cloud block.

[0062] Specifically, the data acquisition device determines the block center corresponding to the point cloud block, and divides the point cloud block into multiple point cloud sub-blocks according to the distance between the block center and each point in the point cloud block, and according to the distance from the block center from near to far or from far to near.

[0063] As can be seen, in this embodiment, the point cloud block is divided by the distance between the block center and the points in the point cloud block. This can reduce the amount of computation and help to group adjacent points in the point cloud block into the same point cloud sub-block, thereby forming multiple continuous regions, so that each region can be processed and analyzed separately in the future.

[0064] In some embodiments, after the data acquisition device collects point cloud data of the scene where the target object is located, it obtains the collected raw point cloud data. Since there may be multiple objects in the scene where the target object is located, the data acquisition device can cluster the raw point cloud data to obtain each point cloud block, and determine the point cloud block of the target object from each point cloud block. The cluster center corresponding to the point cloud sub-block determined in the clustering process is the block center mentioned in the above embodiments.

[0065] In some embodiments, the coordinates of each point can be added to the coordinates of other points in the same cluster, and the summed coordinates can be divided by the number of points in the same cluster to obtain the coordinates of the cluster center, thereby determining the location of the cluster center.

[0066] In some embodiments, the step "dividing the point cloud block into multiple point cloud sub-blocks based on the distances between the block center and each point in the point cloud block" specifically includes, but is not limited to, the following: determining the maximum distance from the distances between the block center and each point in the point cloud block; determining multiple radii with the block center as the geometric center based on the maximum distance; having multiple radii less than or equal to the maximum distance; and dividing the point cloud block into multiple point cloud sub-blocks based on the multiple radii.

[0067] Specifically, the data acquisition device determines the maximum distance from the block center to each point in the point cloud block. Based on the maximum distance, the data acquisition device determines multiple radii with the block center as the geometric center. The data acquisition device can then sequentially determine the current radius from these multiple radii according to their size order, extract the corresponding points from the point cloud block, and use the set of extracted points as the point cloud sub-block corresponding to the current radius.

[0068] In some embodiments, the geometric center can be the center of at least one plane, such as the center of a circle within at least one plane. When the geometric center is a circle, the corresponding radii refer to the radii of the circles. For each plane, the data acquisition object can determine the current plane radius sequentially from the multiple radii according to their size order, extract the corresponding points from the point cloud block, and use the set of extracted points as the point cloud corresponding to the current plane radius under that plane. The data acquisition device can treat the point clouds corresponding to the same current plane radius under different planes as a whole to obtain the point cloud sub-block corresponding to that current plane radius.

[0069] In other embodiments, the geometric center can be the center of a solid, such as the center of a sphere. When the geometric center is the center of a sphere, the corresponding multiple radii refer to the radii of the sphere. The data acquisition device can determine the current sphere radius sequentially from the multiple sphere radii according to their size order, extract the corresponding points from the point cloud block, and use the set of extracted points as the point cloud sub-block corresponding to the current sphere radius.

[0070] In some embodiments, the multiple radii include multiple sphere radii with the block center as the sphere center. The step "dividing the point cloud block into multiple point cloud sub-blocks according to the multiple radii" specifically includes, but is not limited to, determining the point cloud sub-block corresponding to each current sphere radius among the multiple sphere radii.

[0071] The point cloud sub-block corresponding to the current sphere radius is composed of points located within the region corresponding to the current sphere radius but not within the region corresponding to the reference sphere radius; the reference sphere radius is the sphere radius smaller than the current sphere radius among multiple sphere radii.

[0072] Specifically, the data acquisition device can determine the current sphere radius from multiple sphere radii in order of size, extract the corresponding points from the point cloud block, and use the set of extracted points as the point cloud sub-block corresponding to the current sphere radius.

[0073] As can be seen, in this embodiment, a point cloud block can be considered as a sphere, and the center of the block can be considered as the center of the sphere. The corresponding point cloud sub-blocks are then determined from the point cloud block based on the radius of each sphere. Since points closer to the block center are less likely to be noise, and points farther from the block center are more likely to be noise, the above-described point cloud sub-block division method clearly identifies which point cloud sub-blocks contain points less likely to be noise and which contain points more likely to be noise. This provides a reference for subsequent object classification, thereby further improving the accuracy of object classification.

[0074] In some embodiments, such as Figure 3As shown, all points on the left constitute the point cloud block of the target object. Within this point cloud block, the maximum distance between the block center and each point in the point cloud block is determined as the maximum sphere radius R. Based on the maximum sphere radius R, spheres with radii of R / 3, 2R / 3, and R are constructed respectively, dividing the point cloud block into three point cloud sub-blocks, namely C1, C2, and C3.

[0075] In some embodiments, the data acquisition device can determine the current sphere radius sequentially from multiple sphere radii in ascending order. For each current sphere radius, the data acquisition device assigns points located within the region corresponding to the current sphere radius that have not yet been assigned to a point cloud sub-block to the point cloud sub-block corresponding to the current sphere radius.

[0076] In some embodiments, the data acquisition device can also determine the current sphere radius sequentially from multiple sphere radii in descending order. The data acquisition device can first divide the points located within the region corresponding to the current sphere radius into different point cloud sub-blocks, and remove duplicate points in the initially divided point cloud sub-blocks to obtain the final divided point cloud sub-blocks.

[0077] It should be noted that the methods for dividing point cloud sub-blocks are not limited to the above-mentioned methods. Those skilled in the art can also determine the current sphere radius from multiple sphere radii in a non-sequential manner according to actual needs, so as to determine the corresponding point cloud sub-blocks based on the current sphere radius.

[0078] In some embodiments, each point in the object sub-region has a corresponding regional feature. Step S206 specifically includes, but is not limited to, determining the association relationship between points in the object sub-region based on the regional features corresponding to each point in the object sub-region for each object sub-region, thereby obtaining a first association feature.

[0079] The regional features corresponding to each point include its position, motion, and signal-to-noise ratio (SNR) characteristics in three-dimensional space. Position features include the distance and angle between each point and the data acquisition device, such as the pitch angle, used to provide information about the point's position in three-dimensional space. Motion features include the velocity information of each point in three-dimensional space, and the SNR characteristics include the signal-to-noise ratio information of each point in three-dimensional space.

[0080] Specifically, for each object sub-region, the data acquisition device obtains the association relationship between points in the object sub-region based on the difference between the regional features corresponding to each point in the object sub-region, and then determines the first association feature between points based on the association relationship.

[0081] As can be seen, in this embodiment of the application, by determining the first association feature between each object sub-region through the difference of regional features between each point in each object sub-region, more fine-grained feature information can be obtained in each object sub-region, thereby enhancing the discriminativeness of point cloud features.

[0082] In some embodiments, it is assumed that the regional features corresponding to each point in each object sub-region include the distance D1 of each point from the data acquisition device, the angle D2 of each point from the data acquisition device, the velocity D3 of each point, and the signal-to-noise ratio D4 of each point. For a certain object sub-region, the correlation between any two points in the object sub-region can be the difference between D1, D2, D3, and D4 between the two points. Combining these four differences yields the first correlation feature between the two points.

[0083] In some embodiments, the region features include first location information; the first location information is the location information of each point in the object sub-region relative to the same reference object. The step "For each object sub-region, determine the association relationship between points in the object sub-region based on the region features corresponding to each point in the object sub-region, and obtain a first association feature" specifically includes, but is not limited to, determining, for each object sub-region, second location information between points in the object sub-region based on the first location information corresponding to each point in the object sub-region; and obtaining the first association feature based on the second location information.

[0084] In some embodiments, the reference object may be a data acquisition device, and the first location information includes the relative orientation, distance, and relative angle between each point and the same data acquisition device. For each object sub-region, the data acquisition device determines the distance between points in the object sub-region based on the first location information corresponding to each point in the object sub-region, thereby obtaining the second location information. The data acquisition device may directly use the second location information as the first association feature, or it may use it together with other feature information as the first association feature; this application does not limit this.

[0085] Specifically, for each object sub-region, the data acquisition device determines the distance between points in the object sub-region based on the first location information corresponding to each point in the object sub-region, thus obtaining the second location information. The data acquisition device can directly use the second location information as the first association feature, or it can use it together with other feature information as the first association feature.

[0086] As can be seen, in this embodiment of the application, by capturing the positions of each point within each object sub-region, the spatial relationships and contextual information between these points can be captured. This helps to better understand the local structure and regional features within each object sub-region, thereby improving the accuracy of classification.

[0087] In some embodiments, the first positional information is the distance of each point in the object sub-region relative to the same reference object; the second positional information is the distance between points in the object sub-region. The step "For each object sub-region, determine the second positional information between points in the object sub-region based on the first positional information corresponding to each point in the object sub-region" specifically includes, but is not limited to: for each object sub-region, aligning each point with the reference object according to the first positional information corresponding to each point in the object sub-region to obtain the coordinates of each point relative to the reference object; and determining the distance between points in the object sub-region based on the coordinate differences between points in the object sub-region.

[0088] Specifically, for each object sub-region, the data acquisition device aligns each point with the reference object based on its first position information within the sub-region, obtaining the coordinates of each point relative to the reference object. The data acquisition device then groups every two points within the object sub-region and calculates the difference between the coordinates of points within each group, obtaining the coordinate difference. Finally, it calculates the distance between points within each group based on the coordinate differences between them.

[0089] In some embodiments, step S208 specifically includes, but is not limited to, the following: for each object sub-region, based on the region features corresponding to each point in the object sub-region and each point in the adjacent object sub-regions, determine the association relationship between each point in the object sub-region and each point in the adjacent object sub-regions, and obtain a second association feature.

[0090] Specifically, for each object sub-region, the data acquisition device obtains the association relationship between each point in the object sub-region and each point in the adjacent object sub-region based on the difference between the regional features corresponding to each point in the object sub-region and each point in the adjacent object sub-region. Based on the association relationship, the second association feature between points in two adjacent object sub-regions is determined.

[0091] As can be seen, in this embodiment of the application, by determining the difference in regional features between each point in each object sub-region and adjacent object sub-regions, the second association feature between each two adjacent object sub-regions can be obtained, thus obtaining the association between each object sub-region and extracting more comprehensive and richer feature information, thereby strengthening the association of point cloud features.

[0092] In some embodiments, for each object sub-region, the data acquisition device determines the distance between each point in the object sub-region and each point in the adjacent object sub-region based on the first location information corresponding to each point in each object sub-region, as a second association feature.

[0093] In some embodiments, the step "for each object sub-region, based on the region features corresponding to each point in the object sub-region and each point in the adjacent object sub-region, determine the association relationship between each point in the object sub-region and each point in the adjacent object sub-region, and obtain the second association feature" specifically includes, but is not limited to: for each object sub-region, based on the first location information corresponding to each point in the object sub-region and each point in the adjacent object sub-region, determine the third location information between each point in the object sub-region and each point in the adjacent object sub-region; and obtain the second association feature based on the third location information.

[0094] Specifically, for each object sub-region, the data acquisition device determines the distance between each point in the object sub-region and each point in the adjacent object sub-regions based on the first location information corresponding to each point in the object sub-region and the adjacent object sub-regions, thus obtaining the third location information. The data acquisition device can directly use the third location information as the second association feature, or it can use the third location information together with other feature information as the second association feature.

[0095] As can be seen, in this embodiment of the application, by determining the second association feature between each two adjacent object sub-regions through the distance difference between each point in each object sub-region and adjacent object sub-regions, the association between each object sub-region can be obtained, and more comprehensive and richer feature information can be extracted, thereby strengthening the association of point cloud features.

[0096] In some embodiments, the third location information is the distance between each point in the object sub-region and each point in the adjacent object sub-regions. The step "For each object sub-region, determine the third location information between each point in the object sub-region and each point in the adjacent object sub-regions based on the first location information corresponding to each point in the object sub-region and each point in the adjacent object sub-regions" specifically includes, but is not limited to: for each object sub-region, aligning the coordinates of each point in the object sub-region and each point in the adjacent object sub-regions with a reference object based on the first location information corresponding to each point in the object sub-region and each point in the adjacent object sub-regions, obtaining the coordinates of each point relative to the reference object; and determining the distance between each point in the object sub-region and each point in the adjacent object sub-regions based on the coordinate differences between each point in the object sub-region and each point in the adjacent object sub-regions.

[0097] Specifically, for each object sub-region, the data acquisition device aligns the coordinates of each point in the object sub-region and each point in the adjacent object sub-regions with the reference object, based on the first position information of each point in the object sub-region and each point in the adjacent object sub-regions, thus obtaining the coordinates of each point relative to the reference object. The coordinate difference is then calculated based on the coordinate differences between each point in the object sub-region and each point in the adjacent object sub-regions. Finally, the distance between each point in the object sub-region and each point in the adjacent object sub-regions is calculated based on these multiple coordinate differences.

[0098] In some embodiments, step S210 specifically includes, but is not limited to: weighting and fusing the first associated feature and the second associated feature according to the first weight corresponding to the first associated feature and the second weight corresponding to the second associated feature to obtain a fused feature; performing dimensionality reduction processing on the fused feature to obtain a target feature; and determining the category of the target object according to the target feature.

[0099] Weighted fusion refers to fusing different features according to certain weights to obtain more accurate features, i.e., fused features.

[0100] Specifically, the data acquisition device performs weighted fusion of the first associated feature and the second associated feature based on the first weight corresponding to the first associated feature and the second weight corresponding to the second associated feature to obtain a fused feature. The data acquisition device then performs dimensionality reduction processing on the fused feature to obtain the target feature, and determines the category of the target object based on the target feature.

[0101] As can be seen, in this embodiment, by weighted fusion of the first and second associated features, the advantages of two different scale features can be combined to simultaneously enhance the discriminativeness and correlation of point cloud features, thereby improving the discriminativeness of target object classification and thus increasing the accuracy of object classification. Since the dimensionality of the fused features may be relatively large, this could potentially affect the classification speed. Therefore, it is necessary to reduce the dimensionality of the fused features to improve the classification speed.

[0102] In some embodiments, the fused features can be dimensionality reduced by at least one of Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), or Convolutional Neural Network (CNN) to obtain the target features.

[0103] In some embodiments, such as Figure 3 As shown, after dividing the object region into three sub-regions C1, C2, and C3, the first association feature between each point in C1 is extracted to obtain... Extracting the first association features between each point in C2 yields... Extracting the first association features between each point in C3 yields... Extracting the second association features between each point in C1 and each point in C2 yields... Extract the second association features between each point in C2 and each point in C3 to obtain .

[0104] In some embodiments, the fusion features can be obtained by processing using the following formula (1):

[0105] (1)

[0106] Where F represents the fusion feature, Indicates the first associated feature The corresponding first weight, Indicates the first associated feature The corresponding first weight, Indicates the first associated feature The corresponding first weight, Indicates the second associated feature The corresponding second weight, Indicates the second associated feature The corresponding second weight.

[0107] In some embodiments, the weights of the first association features corresponding to different object sub-regions can be calculated based solely on the number of points in each object sub-region. Therefore, in formula (1) , and The values ​​are respectively the ratios of the number of points in C1, C2, and C3 to the number of points in the target point cloud, while and The values ​​can all be: Here, the number of target point clouds refers to the total number of points in the object region. It should be noted that the first and second weights mentioned above can be averaged or calculated using an adaptive reinforcement learning (AdaBoost) algorithm; this application does not impose any specific restrictions on this.

[0108] As can be seen, in this embodiment of the application, by weighted fusion of associated features at different scales, the local and global features of the point cloud can be fully integrated, which not only enhances the discriminativeness and distinguishability of the target features, but also improves the accuracy of classification and recognition.

[0109] In some embodiments, the step "weightedly fusing the first associated feature and the second associated feature according to the first weight corresponding to the first associated feature and the second weight corresponding to the second associated feature to obtain a fused feature" specifically includes, but is not limited to: determining the first weight corresponding to the first associated feature; determining the second weight corresponding to the second associated feature; and weightedly fusing the first associated feature according to the first weight and the second weight to obtain a fused feature.

[0110] The first weight is positively correlated with at least one of the number of points or the center distance; the number of points is the number of points in the object sub-region on which the first association feature is extracted; the center distance is the distance from each point in the object sub-region to the center of the block corresponding to the object region; the second weight is negatively correlated with the first weight.

[0111] In other words, the higher the weight of the first local feature corresponding to an object sub-region with a larger number of points, the higher the weight of the second local feature corresponding to two adjacent object sub-regions with a larger center distance. Since the sum of all weights is 1, the higher the weight of the first local feature, the lower the weight of the second local feature.

[0112] As can be seen, in this embodiment, since the number of points in an object sub-region reflects the information density within that sub-region, a greater number of points means richer feature information within that sub-region. To fully utilize all information, this embodiment assigns higher weights to object sub-regions with more points, which improves the accuracy of subsequent object classification. Furthermore, since this embodiment divides the object region into sub-regions using different sphere radii, some sub-regions will be closer to the block center, while others will be farther away. Since points closer to the block center are less likely to be noise, while points farther away are more likely, assigning higher weights to the first associated features of the object sub-regions corresponding to points closer to the block center minimizes noise interference, thereby ensuring the accuracy of subsequent classification and recognition.

[0113] In some embodiments, the category of the target object is obtained by inputting the object region of the target object into an object classification model for classification and recognition. The training steps of the object classification model include: acquiring each sample object region; each sample object region having corresponding object category label information; dividing the sample object region into multiple sample object sub-regions using an initial object classification model; determining a first sample association feature based on the association relationship between points in each sample object sub-region; determining a second sample association feature based on the association relationship between points in each sample object sub-region and points in adjacent object sub-regions; predicting the object category corresponding to the sample object region based on the first and second sample association features to obtain the category prediction result; and adjusting the model parameters of the initial object classification model based on the difference between the category prediction result and the object category label information to train the model and obtain the object classification model.

[0114] Among them, the category label information is used to characterize the category of the object to which the corresponding sample object region belongs.

[0115] Specifically, the data acquisition device acquires the regions of each sample object. The data acquisition device divides the sample object regions into multiple sample object sub-regions using an initial object classification model. Based on the relationships between points in each sample object sub-region, a first sample association feature is determined. The initial object classification model determines a second sample association feature based on the relationships between points in each sample object sub-region and points in adjacent object sub-regions. Based on the first and second sample association features, the initial object classification model predicts the object category corresponding to the sample object region, obtaining the category prediction result. Based on the difference between the category prediction result and the object category label information, the initial object classification model adjusts its model parameters for model training, resulting in a trained object classification model.

[0116] It should be noted that both the sample object region and the object region refer to regions obtained by collecting data on a specific object. The sample object region is the region obtained by collecting data on a specific object during the model training phase, while the object region is the region obtained by collecting data on a specific object during the object classification phase.

[0117] In some embodiments, the process of training the object classification model is a continuous process. That is, after the data acquisition device classifies a specific object using the trained object classification model, it can continue to train the trained object classification model based on the object classification result, thereby achieving the goal of continuously optimizing the object classification model.

[0118] In some embodiments, the initial object classification model can further acquire the region features corresponding to each point in each sample object sub-region, so as to determine the association relationship between points in each sample object sub-region based on the region features, thereby obtaining the first sample association features. The initial object classification model then determines the association relationship between each point in the object sub-region and each point in adjacent object sub-regions based on the region features, thereby obtaining the second sample association features. Subsequently, the initial object classification model performs weighted fusion and dimensionality reduction on the first and second sample association features to obtain the target sample features, and predicts the category corresponding to the sample object region based on the target sample features, thereby obtaining the category prediction result.

[0119] It should be noted that each point in each sample object sub-region corresponds to object category labeling information and region features, which can be represented as follows: ,in, It labels the object category information and region features corresponding to a single point, including the category of the object to which the point belongs, the distance of the point from the reference object, the pitch angle of the point from the reference object, the velocity of the point, and the signal-to-noise ratio.

[0120] In some embodiments, the object classification model can also be trained using methods such as Support Vector Machine (SVM), Logistic Regression (LR), and Convolutional Neural Network (CNN).

[0121] In some embodiments, the point cloud acquisition device in the data acquisition equipment includes a millimeter-wave radar device. Millimeter-wave radar devices are commonly used in smart homes, autonomous driving, and intelligent transportation systems. In smart home devices, millimeter-wave radar devices can classify adults, children, pets, and other objects such as robot vacuums within the home, thereby enabling intelligent management and control of the home environment.

[0122] In some embodiments, the categories collected by the data acquisition device include a first category. Following S210, the object classification method of this application embodiment further includes: sending the first category to a smart home device to instruct the smart home device to perform corresponding device control processing according to the first category; receiving a second category sent by the smart home device; and, if the first category and the second category are inconsistent, performing calibration processing on the first category according to the second category.

[0123] The second category is obtained by smart home devices capturing images of the target object and then classifying and recognizing the captured images.

[0124] Specifically, the data acquisition device can send a first category to the smart home device, instructing the smart home device to perform corresponding device control processing based on the first category. After performing the corresponding device control processing based on the first category, the smart home device can also capture images of the target object through a camera to classify and identify the second category to which the target object belongs. If the first category and the second category are inconsistent, the smart home device can send the second category to the data acquisition device, so that the data acquisition device can calibrate its own identified first category based on the second category. If the first category and the second category are consistent, the smart home device will not perform information transmission processing, or will send a category consistency prompt message to the data acquisition device.

[0125] As can be seen, in this embodiment, the data acquisition device can be linked with smart home devices, and the smart home devices can classify and identify target objects, thereby calibrating the classification results of the data acquisition device in a timely manner and improving the accuracy of subsequent object classification.

[0126] For example, the point cloud acquisition device in the data acquisition equipment is a millimeter-wave radar device, and the smart home device is a smart pet feeder, which is equipped with a controllable light and a camera. When the millimeter-wave radar device identifies the first category of the target object, it sends this first category to the smart pet feeder. If the first category is a person and the smart pet feeder detects that it is nighttime, the smart pet feeder automatically turns on the light to facilitate feeding the pet. If the first category is a pet and the smart pet feeder detects that it is nighttime, the smart pet feeder turns off the light to save energy. Subsequently, the smart pet feeder can also acquire images of the target object through the camera and classify the acquired images to obtain a second category. If the first category is a person and the second category is a pet, or vice versa, it indicates that the millimeter-wave radar device may make a recognition error. The smart pet feeder can then feed this result back to the millimeter-wave radar device, allowing the millimeter-wave radar device to calibrate the classification result.

[0127] In some embodiments, the sample object region is a sample point cloud block, the sample object sub-region is a sample point cloud sub-block, the object region is a point cloud block, and the object sub-region is a point cloud sub-block. The object classification method of this application further includes, but is not limited to, the following steps:

[0128] (1) Obtain the point cloud blocks of each sample.

[0129] (2) The sample point cloud block is divided into multiple sample point cloud sub-blocks by the initial object classification model.

[0130] (3) Determine the first sample association features based on the relationship between points in each sample point cloud sub-block.

[0131] (4) Determine the association features of the second sample based on the association relationship between the points in each sample point cloud sub-block and the points in the adjacent point cloud sub-blocks.

[0132] (5) Based on the first sample association features and the second sample association features, predict the object category corresponding to the sample point cloud block and obtain the category prediction result.

[0133] (6) Based on the difference between the category prediction results and the object category label information, adjust the model parameters of the initial object classification model to train the model and obtain the object classification model.

[0134] (7) Obtain the point cloud block of the target object and input the point cloud block into the trained object classification model so that the object classification model can perform subsequent steps (8) to (23).

[0135] (8) Determine the block center corresponding to the point cloud block.

[0136] (9) Determine the maximum distance from the distance between the block center and each point in the point cloud block.

[0137] (10) Determine the radii of multiple spheres with the block center as the sphere center based on the maximum distance.

[0138] (11) For each current sphere radius among multiple sphere radii, determine the point cloud sub-block corresponding to the current sphere radius from the point cloud block.

[0139] (12) For each point cloud sub-block, based on the first position information corresponding to each point in the point cloud sub-block, each point is aligned with the reference object to obtain the coordinates of each point relative to the reference object.

[0140] (13) Determine the distance between points in the point cloud sub-block based on the coordinate difference between points in the point cloud sub-block, and obtain the second position information.

[0141] (14) Obtain the first associated feature based on the second location information.

[0142] (15) For each point cloud sub-block, based on the first position information corresponding to each point in the point cloud sub-block and the adjacent point cloud sub-block, the coordinates of each point in the point cloud sub-block and the adjacent point cloud sub-block are aligned with the reference object to obtain the coordinates of each point relative to the reference object.

[0143] (16) Based on the coordinate difference between each point in the point cloud sub-block and each point in the adjacent point cloud sub-block, determine the distance between each point in the point cloud sub-block and each point in the adjacent point cloud sub-block to obtain the third position information.

[0144] (17) Obtain the second associated feature based on the third location information.

[0145] (18) Based on the first weight corresponding to the first associated feature and the second weight corresponding to the second associated feature, the first associated feature and the second associated feature are weighted and fused to obtain the fused feature.

[0146] (19) Dimensionality reduction is performed on the fusion features to obtain the target features.

[0147] (20) Determine the category of the target object based on the target characteristics to obtain the first category.

[0148] (21) Send the first category to the smart home device to instruct the smart home device to perform the corresponding device control processing according to the first category.

[0149] (22) Receiving the second category of data sent by smart home devices.

[0150] (23) In the case of inconsistency between the first category and the second category, the first category shall be calibrated according to the second category.

[0151] It should be noted that, in addition to the feature extraction and classification methods mentioned in the above embodiments, the embodiments of this application can also perform target feature extraction and classification based on statistical feature extraction, deep convolutional neural networks, and machine learning feature extraction and classification. Furthermore, besides using the multi-scale feature extraction method (i.e., the first associated feature and the second associated feature) mentioned in the above embodiments, multi-scale features can also be extracted using a multi-scale feature extraction method based on deep convolutional networks.

[0152] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0153] Based on the same inventive concept, this application also provides an object classification device. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more object classification device embodiments provided below can be found in the limitations of the object classification method above, and will not be repeated here.

[0154] like Figure 4 As shown, this application embodiment provides an object classification device, including:

[0155] Module 402 is used to obtain the object region of the target object;

[0156] The partitioning module 404 is used to divide the object region of the target object into multiple object sub-regions;

[0157] The feature extraction module 406 is used to determine the first association feature based on the relationship between points in each object sub-region; and to determine the second association feature based on the relationship between points in each object sub-region and points in adjacent object sub-regions.

[0158] The classification and recognition module 408 is used to determine the category of the target object based on the first association feature and the second association feature.

[0159] The aforementioned object classification device acquires the object region of a target object; divides the object region into multiple object sub-regions; and determines a first association feature based on the relationships between points in each object sub-region. This allows for the extraction of finer-grained feature information from object sub-regions at different scales, enhancing the discriminative power of point features and facilitating subsequent classification tasks. A second association feature is determined based on the relationships between points in each object sub-region and points in adjacent object sub-regions. This strengthens the correlation between points in object sub-regions at different scales, enabling the extraction of more comprehensive and richer feature information. Furthermore, determining the target object category based on the first and second association features allows for a thorough fusion of these features, simultaneously enhancing the discriminative power and correlation of point features, thereby improving the discriminative power of target object classification and ultimately increasing the accuracy of object classification.

[0160] In some embodiments, the object region of the target object includes point cloud blocks, and the object sub-region includes point cloud sub-blocks. The partitioning module 404 is further configured to determine the block center corresponding to the point cloud block of the target object; and to divide the point cloud block into multiple point cloud sub-blocks according to the distance between the block center and each point in the point cloud block.

[0161] In some embodiments, the partitioning module 404 is further configured to determine the maximum distance from the distances between the block center and each point of the point cloud block; determine multiple radii with the block center as the geometric center based on the maximum distance; the multiple radii are less than or equal to the maximum distance; and partition the point cloud block into multiple point cloud sub-blocks based on the multiple radii.

[0162] In some embodiments, the plurality of radii include a plurality of sphere radii with the block center as the sphere center. The partitioning module 404 is further configured to determine a point cloud sub-block corresponding to the current sphere radius from the point cloud block for each current sphere radius among the plurality of sphere radii; wherein the point cloud sub-block corresponding to the current sphere radius is composed of points located within the region corresponding to the current sphere radius but not within the region corresponding to the reference sphere radius; the reference sphere radius is a sphere radius smaller than the current sphere radius among the plurality of sphere radii.

[0163] In some embodiments, each point in the object sub-region has a corresponding regional feature. The feature extraction module 406 is further configured to determine the association relationship between points in the object sub-region based on the regional features corresponding to each point in the object sub-region for each object sub-region, thereby obtaining a first association feature.

[0164] In some embodiments, the region features include first location information; the first location information is the location information of each point in the object sub-region relative to the same reference object. The feature extraction module 406 is further configured to, for each object sub-region, determine second location information between points in the object sub-region based on the first location information corresponding to each point in the object sub-region; and obtain a first association feature based on the second location information.

[0165] In some embodiments, the first location information is the distance of each point in the object sub-region relative to the same reference object; the second location information is the distance between points in the object sub-region. The feature extraction module 406 is further configured to, for each object sub-region, align the coordinates of each point with the reference object according to the first location information corresponding to each point in the object sub-region, to obtain the coordinates of each point relative to the reference object; and determine the distance between points in the object sub-region based on the coordinate differences between points in the object sub-region.

[0166] In some embodiments, the feature extraction module 406 is further configured to, for each object sub-region, determine the association relationship between each point in the object sub-region and each point in the adjacent object sub-region based on the region features corresponding to each point in the object sub-region and the adjacent object sub-region, thereby obtaining a second association feature.

[0167] In some embodiments, the feature extraction module 406 is further configured to, for each object sub-region, determine the third position information between each point in the object sub-region and each point in the adjacent object sub-region based on the first position information corresponding to each point in the object sub-region and the adjacent object sub-region; and obtain the second associated feature based on the third position information.

[0168] In some embodiments, the third location information is the distance between each point in the object sub-region and each point in the adjacent object sub-regions. The feature extraction module 406 is further configured to, for each object sub-region, align the coordinates of each point in the object sub-region and the adjacent object sub-regions with the reference object based on the first location information corresponding to each point in the object sub-region and the adjacent object sub-regions, to obtain the coordinates of each point relative to the reference object; and determine the distance between each point in the object sub-region and each point in the adjacent object sub-regions based on the coordinate difference between each point in the object sub-region and each point in the adjacent object sub-regions.

[0169] In some embodiments, the classification and recognition module 408 is further configured to perform weighted fusion of the first associated feature and the second associated feature according to the first weight corresponding to the first associated feature and the second weight corresponding to the second associated feature to obtain fused features; perform dimensionality reduction processing on the fused features to obtain target features; and determine the category of the target object according to the target features.

[0170] In some embodiments, the classification and recognition module 408 is further configured to determine a first weight corresponding to a first associated feature; the first weight is positively correlated with at least one of the number of points or the center distance; the number of points is the number of points in the object sub-region on which the first associated feature is extracted; the center distance is the distance from each point in the object sub-region to the center of the block corresponding to the object region; determine a second weight corresponding to a second associated feature; the second weight is negatively correlated with the first weight; and perform weighted fusion of the first associated feature according to the first weight and the second weight to obtain a fused feature.

[0171] In some embodiments, the category of the target object is obtained by inputting the object region of the target object into an object classification model for classification and recognition. The object classification apparatus of this application embodiment further includes a model training module, used to acquire each sample object region; each sample object region has corresponding object category label information; divide the sample object region into multiple sample object sub-regions using an initial object classification model; determine a first sample association feature based on the association relationship between points in each sample object sub-region; determine a second sample association feature based on the association relationship between points in each sample object sub-region and points in adjacent object sub-regions; predict the object category corresponding to the sample object region based on the first and second sample association features to obtain a category prediction result; and adjust the model parameters of the initial object classification model based on the difference between the category prediction result and the object category label information to perform model training and obtain an object classification model.

[0172] In some embodiments, the category of the target object includes a first category. The object classification apparatus of this application embodiment further includes a category calibration module, configured to send the first category to a smart home device to instruct the smart home device to perform corresponding device control processing according to the first category; receive a second category sent by the smart home device; the second category is obtained by the smart home device after capturing an image of the target object and classifying and recognizing the captured image; and, if the first category and the second category are inconsistent, perform calibration processing on the first category according to the second category.

[0173] In some embodiments, a computer device is provided, which may be a data acquisition device, and its internal structure diagram may be as follows: Figure 5 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the object classification method described above.

[0174] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] In some embodiments, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0176] In some embodiments, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the steps in the above method embodiments.

[0177] In some embodiments, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0178] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0179] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An object classification method, characterized in that, include: Obtain the object region of the target object; the target object includes any one or more of adults, children, pets, and robot vacuum cleaners; Determine the block center corresponding to the point cloud block included in the object region of the target object; divide the point cloud block into multiple point cloud sub-blocks included in the object sub-region based on the distance between the block center and each point in the point cloud block; Based on the relationship between points in each of the object sub-regions, a first association feature is determined; The second association feature is determined based on the association relationship between points in each of the object sub-regions and points in adjacent object sub-regions; A first weight is determined corresponding to the first associated feature; the first weight is positively correlated with at least one of the number of points or the center distance; the number of points is the number of points in the object sub-region on which the first associated feature is extracted; the center distance is the distance from each point in the object sub-region to the center of the block corresponding to the object region. Determine the second weight corresponding to the second associated feature; the second weight is negatively correlated with the first weight. The first associated feature is weighted and fused according to the first weight and the second weight to obtain the fused feature; The fusion features are then subjected to dimensionality reduction processing to obtain the target features; The category of the target object is determined based on the target characteristics.

2. The method according to claim 1, characterized in that, The step of dividing the point cloud block into multiple point cloud sub-blocks based on the distances between the block center and each point in the point cloud block includes: Determine the maximum distance from the distances between the center of the block and each point of the point cloud block; Based on the maximum distance, determine multiple radii with the block center as the geometric center; the multiple radii are less than or equal to the maximum distance; The point cloud block is divided into multiple point cloud sub-blocks based on the multiple radii.

3. The method according to claim 2, characterized in that, The plurality of radii includes the radii of multiple spheres with the center of the block as the center; the step of dividing the point cloud block into multiple point cloud sub-blocks according to the plurality of radii includes: For each current sphere radius among the plurality of sphere radii, determine the point cloud sub-block corresponding to the current sphere radius from the point cloud block; The point cloud sub-block corresponding to the current sphere radius is composed of points located within the region corresponding to the current sphere radius but not within the region corresponding to the reference sphere radius; the reference sphere radius is the sphere radius smaller than the current sphere radius among multiple sphere radii.

4. The method according to claim 1, characterized in that, Each point in the object sub-region has a corresponding regional feature; determining the first association feature based on the association relationship between points in each object sub-region includes: For each object sub-region, the association relationship between points in the object sub-region is determined based on the regional features corresponding to each point in the object sub-region, thus obtaining the first association feature.

5. The method according to claim 4, characterized in that, The region features include first location information; the first location information is the location information of each point in the object sub-region relative to the same reference object; for each object sub-region, the association relationship between points in the object sub-region is determined based on the region features corresponding to each point in the object sub-region, resulting in a first association feature, including: For each object sub-region, the second position information between points in the object sub-region is determined based on the first position information corresponding to each point in the object sub-region. The first associated feature is obtained based on the second location information.

6. The method according to claim 5, characterized in that, The first position information is the distance of each point in the object sub-region relative to the same reference object; the second position information is the distance between points in the object sub-region; determining the second position information between points in the object sub-region based on the first position information corresponding to each point in the object sub-region for each object sub-region includes: For each object sub-region, based on the first position information corresponding to each point in the object sub-region, each point is aligned with the reference object to obtain the coordinates of each point relative to the reference object; The distance between points in the object sub-region is determined based on the coordinate difference between points in the object sub-region.

7. The method according to claim 1, characterized in that, The step of determining the second association feature based on the association relationship between points in each object sub-region and points in adjacent object sub-regions includes: For each object sub-region, based on the region features corresponding to each point in the object sub-region and each point in the adjacent object sub-regions, the association relationship between each point in the object sub-region and each point in the adjacent object sub-regions is determined, thus obtaining the second association feature.

8. The method according to claim 7, characterized in that, For each object sub-region, based on the region features corresponding to each point in the object sub-region and each point in the adjacent object sub-regions, the association relationship between each point in the object sub-region and each point in the adjacent object sub-regions is determined, resulting in a second association feature, including: For each object sub-region, based on the first position information corresponding to each point in the object sub-region and each point in the adjacent object sub-regions, the third position information between each point in the object sub-region and each point in the adjacent object sub-regions is determined. The second associated feature is obtained based on the third location information.

9. The method according to claim 8, characterized in that, The third location information is the distance between each point in the object sub-region and each point in the adjacent object sub-regions; the step of determining the third location information between each point in the object sub-region and each point in the adjacent object sub-regions for each object sub-region, based on the first location information corresponding to the object sub-region and each point in the adjacent object sub-regions, includes: For each object sub-region, based on the first position information corresponding to each point in the object sub-region and the adjacent object sub-regions, the coordinates of each point in the object sub-region and the adjacent object sub-regions are aligned with the reference object to obtain the coordinates of each point relative to the reference object. The distance between each point in the object sub-region and each point in the adjacent object sub-region is determined based on the coordinate difference between each point in the object sub-region and each point in the adjacent object sub-region.

10. The method according to claim 1, characterized in that, The category of the target object is obtained by inputting the object region of the target object into an object classification model for classification and recognition; the training steps of the object classification model include: Obtain the region of each sample object; each sample object region has corresponding object category labeling information; The sample object region is divided into multiple sample object sub-regions using an initial object classification model; Based on the relationship between points in each of the sample object sub-regions, the first sample association feature is determined; The second sample association feature is determined based on the association relationship between points in each sample object sub-region and points in adjacent object sub-regions; Based on the first sample association features and the second sample association features, the object category corresponding to the sample object region is predicted to obtain the category prediction result; Based on the difference between the category prediction result and the object category label information, the model parameters of the initial object classification model are adjusted to train the model and obtain the object classification model.

11. The method according to any one of claims 1 to 10, characterized in that, The category of the target object includes a first category; after determining the category of the target object based on the first association feature and the second association feature, the method further includes: The first category is sent to the smart home device to instruct the smart home device to perform corresponding device control processing according to the first category; The system receives a second category sent by the smart home device; the second category is obtained by the smart home device after capturing an image of the target object and classifying and recognizing the captured image. If the first category and the second category are inconsistent, the first category shall be calibrated according to the second category.

12. An object classification device, characterized in that, include: The acquisition module is used to acquire the object region of the target object; the target object includes any one or more of adults, children, pets, and robot vacuum cleaners; The segmentation module is used to determine the block center corresponding to the point cloud block included in the object region of the target object; and to divide the point cloud block into multiple point cloud sub-blocks included in the object sub-region based on the distance between the block center and each point in the point cloud block. The feature extraction module is used to determine a first association feature based on the association relationship between points in each object sub-region; and to determine a second association feature based on the association relationship between points in each object sub-region and points in adjacent object sub-regions. A classification and recognition module is used to determine a first weight corresponding to the first associated feature; the first weight is positively correlated with at least one of the number of points or the center distance; the number of points is the number of points in the object sub-region on which the first associated feature is extracted; the center distance is the distance from each point in the object sub-region to the center of the block corresponding to the object region; determine a second weight corresponding to the second associated feature; the second weight is negatively correlated with the first weight; and perform weighted fusion of the first associated feature according to the first weight and the second weight to obtain a fused feature; The fusion features are then subjected to dimensionality reduction processing to obtain the target features; The category of the target object is determined based on the target characteristics.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images