Object recognition method and device, electronic device, computer readable storage medium

Abstract

Embodiments of the present application disclose an object recognition method and device, an electronic device and a computer readable storage medium, which can be applied to the field of traffic or map. The method comprises: performing spatial rasterization processing on point cloud data to obtain a plurality of voxels in the point cloud coordinate range; determining point cloud intensity values in each voxel, and combining point cloud coordinate values and point cloud intensity values in each voxel to form an input matrix; performing convolution processing on the input matrix to obtain a classification matrix and a regression position matrix, wherein the classification matrix represents whether there is an object position point in each voxel, and the regression position matrix contains a center point position in the voxel and an offset of an object position point in the voxel relative to the center point position; determining a position coordinate of the object position point in the voxel according to the classification position matrix and the regression position matrix, and recognizing a corresponding object through the position coordinate of the object position point in the voxel. The present application has higher object recognition efficiency.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, specifically to an object recognition method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Traffic roads are typically equipped with objects such as road signs, streetlights, traffic lights, cameras, and billboards. In some application scenarios, it is necessary to identify these objects.

[0003] In existing technologies, bounding boxes for each object need to be labeled in point cloud data. Then, the point cloud data and bounding box data are input into a network model for training. Finally, the trained network model is used to identify the location of each bounding box from the input point cloud data. However, the process of labeling rod-shaped objects as bounding boxes in existing technologies is cumbersome. Furthermore, the vectorized recognition results for rod-shaped objects are generally required to be line segments, while the prediction results of the network models trained by existing technologies are bounding boxes, which is difficult to meet the object recognition needs of real-world application scenarios.

[0004] Therefore, how to propose a simpler object recognition method is a technical problem that those skilled in the art need to study and solve. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of this application provide an object recognition method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the embodiments of this application, an object recognition method is provided, comprising: performing spatial rasterization processing on point cloud data to obtain multiple voxels within the coordinate range of the point cloud; determining the point cloud intensity value within each voxel, and combining the point cloud coordinate value and the point cloud intensity value within each voxel to form an input matrix; performing convolution processing on the input matrix to obtain a classification matrix and a regression position matrix, wherein the classification matrix characterizes whether there is an object position point within each voxel, and the regression position matrix includes the center point position within the voxel and the offset of the object position point within the voxel relative to the center point position; determining the position coordinates of the object position point within the voxel according to the classification position matrix and the regression position matrix, and identifying the corresponding object by the position coordinates of the object position point within the voxel.

[0007] According to one aspect of the embodiments of this application, an object recognition device is provided, comprising: a rasterization processing module configured to perform spatial rasterization processing on point cloud data to obtain multiple voxels within the coordinate range of the point cloud; an input matrix determination module configured to determine the point cloud intensity value within each voxel, and to combine the point cloud coordinate value and the point cloud intensity value within each voxel to form an input matrix; a convolution processing module configured to perform convolution processing on the input matrix to obtain a classification matrix and a regression position matrix, wherein the classification matrix characterizes whether there is an object position point within each voxel, and the regression position matrix includes the center point position within the voxel and the offset of the object position point within the voxel relative to the center point position; and an object recognition module configured to determine the position coordinates of the object position point within the voxel according to the classification position matrix and the regression position matrix, and to identify the corresponding object by the position coordinates of the object position point within the voxel.

[0008] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the object recognition method as described above.

[0009] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the object recognition method as described above.

[0010] According to one aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the object recognition method as described above.

[0011] In the technical solution provided in the embodiments of this application, multiple voxels within the point cloud range and the point cloud intensity value corresponding to each voxel are obtained by spatial rasterization processing of point cloud data. Then, the input matrix formed by combining the point cloud coordinate values ​​and point cloud intensity values ​​within each voxel is convolved to obtain the classification matrix and the regression position matrix, respectively. Finally, the position coordinates of each object position point within each voxel are determined based on these two matrices, and the object is identified by the position coordinates of each object position point within the voxel. It can be seen that the object recognition in the embodiments of this application is achieved based on the position coordinates of the object position points within each voxel. When recognizing objects such as rods, it is not necessary to recognize and label the three-dimensional bounding box of the object, thereby improving the simplicity of object recognition.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0014] Figure 1 This is a schematic diagram of one implementation environment involved in this application;

[0015] Figure 2 This is a flowchart illustrating an exemplary embodiment of the object recognition method of this application;

[0016] Figure 3 Is Figure 2 A flowchart of another object recognition method proposed based on the illustrated embodiment;

[0017] Figure 4 This is a schematic diagram illustrating an exemplary process for training an object recognition model.

[0018] Figure 5 This is an exemplary flowchart for training an object recognition model;

[0019] Figure 6 This is an example illustration of the effect of using an object recognition model to recognize objects in an application scenario.

[0020] Figure 7 This is another exemplary flowchart for training an object recognition model;

[0021] Figure 8 This is a block diagram illustrating an object recognition device in an exemplary embodiment of this application;

[0022] Figure 9 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0026] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0027] First, it should be noted that this application relates to the field of artificial intelligence (AI). AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0028] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0029] The object recognition methods, object recognition devices, electronic devices, computer-readable storage media, and computer program products proposed in the embodiments of this application relate to the above-mentioned fields of artificial intelligence technology and machine learning technology.

[0030] It should also be noted that Intelligent Traffic System (ITS), also known as Intelligent Transportation System, effectively integrates advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing, strengthening the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, improves the environment, and saves energy.

[0031] The object recognition methods, object recognition devices, electronic devices, computer-readable storage media, and computer program products proposed in the embodiments of this application also relate to the field of intelligent transportation systems, for example. Figure 1 The example implementation environment.

[0032] Please see Figure 1 In an exemplary intelligent transportation system, there are vehicle-mounted terminals 110 and vehicle-mounted servers 120, and a wired or wireless communication connection can be pre-established between the vehicle-mounted terminals 110 and the vehicle-mounted servers 120.

[0033] The vehicle-mounted terminal 110 is equipped with a lidar system to detect the position of objects on the road, such as... Figure 1 The image shows the positions of various objects on a traffic road, allowing for the acquisition of point cloud data containing the coordinate information of these objects. Objects on the traffic road include, for example, road signs, streetlights, traffic lights, cameras, and billboards. It's important to understand that a lidar (Light Detection and Ranging) system is a radar system that uses emitted laser beams to detect the position, speed, and other information of targets. Its working principle involves emitting a detection signal (laser beam) towards the target, then comparing the received signal reflected back from the target (target echo) with the emitted signal to calculate the coordinate information of the target's three-dimensional position points. Continuous scanning of the target by a lidar system yields a series of coordinate information for three-dimensional position points; these three-dimensional position points are called a point cloud, which is essentially a set of point data. The coordinate information corresponding to these three-dimensional position points is called point cloud data. Point clouds generally also contain reflection intensity information; therefore, point cloud data also includes the reflection intensity information of the point cloud. In the embodiments of this application, the coordinate information of the point cloud is referred to as point cloud coordinate values, and the reflection intensity information of the point cloud is referred to as point cloud intensity values.

[0034] The vehicle terminal 110 transmits the collected point cloud data to the vehicle server 120, which performs spatial rasterization processing on the point cloud data to obtain multiple voxels within the point cloud coordinate range and determines the point cloud intensity value within each voxel. Then, the point cloud coordinate values ​​and point cloud intensity values ​​within each voxel are combined to form an input matrix. By performing convolution processing on the input matrix, a classification matrix and a regression position matrix are obtained. The position coordinates of each object location point within the voxel are determined based on the classification position matrix and the regression position matrix, so as to identify the object by the position coordinates of each object location point within the voxel.

[0035] The object recognition implemented in the vehicle server 120 can be further used in other intelligent transportation programs. For example, the vehicle server 120 can return the location information of the recognized objects to the vehicle terminal 110, which can then display the location information of these objects, such as displaying where there are traffic signs, streetlights, traffic lights, cameras, or billboards on the road, to remind the driver to pay attention to driving safety and abide by traffic regulations, thereby achieving the purpose of intelligent driving.

[0036] For example, the vehicle server 120 may have other intelligent transportation programs running in advance, such as a driving warning program. The vehicle server 120 processes the location information of the objects it identifies. When it detects that the vehicle terminal 110 has an alarm situation during driving, it sends an alarm message to the vehicle terminal 110, thereby achieving the purpose of intelligent driving.

[0037] It should be noted that this embodiment does not limit the specific application scenarios of objects identified based on point cloud data. For example, the aforementioned intelligent transportation program can also be applied to other scenarios besides intelligent driving, such as map production. The identified object information on traffic roads can be used to optimize map data and provide better map services for map users.

[0038] It should also be noted that the vehicle-mounted server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This embodiment does not limit the specific form of the vehicle-mounted server 120.

[0039] It's important to understand that cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to the cloud computing business model. These technologies can form resource pools, allowing for on-demand, flexible, and convenient use. Cloud computing is a computing model that distributes computing tasks across a resource pool composed of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in this network are infinitely scalable, readily available, and can be used on demand, expanded at any time, and paid for based on usage. Driven by the development of the internet, real-time data streams, the diversification of connected devices, and the demands of search services, social networks, mobile commerce, and open collaboration, cloud technology has developed rapidly.

[0040] Figure 2 This is a flowchart illustrating an exemplary embodiment of the object recognition method of this application. The method is applicable to... Figure 1 The implementation environment shown is, for example, by Figure 1 The method is specifically executed by the vehicle-mounted terminal 110 or the vehicle-mounted server 120 in the illustrated implementation environment. Of course, this method can also be applied to other implementation environments and executed by terminal devices or server devices in other implementation environments to achieve object recognition based on point cloud data. This embodiment does not limit this.

[0041] like Figure 2 As shown, in an exemplary embodiment, the object recognition method includes the following steps S210 to S240:

[0042] S210 performs spatial rasterization processing on the point cloud data to obtain multiple voxels within the point cloud coordinate range;

[0043] S220, determine the point cloud intensity value within each voxel, and combine the point cloud coordinate value and point cloud intensity value within each voxel to form the input matrix;

[0044] S230, the input matrix is ​​formed by combining the point cloud coordinates and point cloud intensity values ​​in each voxel. By performing convolution processing on the input matrix, a classification matrix and a regression position matrix are obtained. The classification matrix represents whether there is an object position point in each voxel. The regression position matrix contains the center point position in the voxel and the offset of each object position point in the voxel relative to the center point position.

[0045] S240: Determine the position coordinates of each object's position point within the voxel based on the classification position matrix and the regression position matrix, and identify the corresponding object by using the position coordinates of each object's position point within the voxel.

[0046] Specifically, in step S210, point cloud data refers to the coordinate information and reflection intensity information of a series of three-dimensional coordinate points obtained by continuously detecting the position of an object using a lidar. These three-dimensional coordinate points are called point clouds, the coordinate information of the point clouds is called point cloud coordinate values, and the reflection intensity information of the point clouds is called point cloud intensity values. In some embodiments, point cloud data also includes other information characterizing the surface properties of the object, such as type information, etc., which are not limited here.

[0047] The point cloud coordinate range refers to the range of coordinate values ​​formed by the maximum and minimum coordinate values ​​between three-dimensional coordinate points in the point cloud data. For example, the point cloud coordinate range can be represented as "(min_x, max_x), (min_y, max_y), (min_z, max_z)", where min_x represents the minimum coordinate value on the x-axis, max_x represents the maximum coordinate value on the x-axis, min_y represents the minimum coordinate value on the y-axis, max_y represents the maximum coordinate value on the y-axis, min_z represents the minimum coordinate value on the z-axis, and max_z represents the maximum coordinate value on the z-axis.

[0048] Rasterization of point cloud data involves creating a spatial grid within the point cloud coordinate range according to a preset unit spatial component, thereby obtaining multiple voxels within the point cloud coordinate range. Each voxel represents a spatial grid. For example, the preset unit spatial component can be represented as "[dx,dy,dz]", where dx represents the unit component on the x-axis, dy represents the unit component on the y-axis, and dz represents the unit component on the z-axis. For a voxel at position [i,j,k] in space, the spatial range falling within that voxel can be represented as "[min_x+i*dx:min_x+(i+1)*dx,min_y+j*dy:min_y+(j+1)*dy,min_z+k*dz:min_z+(k+1)*dz]". By traversing the point cloud data within this spatial range, the point data within the corresponding voxel can be obtained.

[0049] In step S220, each voxel obtained by rasterizing the point cloud data may contain one or more point data points, or it may contain zero point data points. That is, a voxel may contain no point data points at all, depending on the actual application scenario. For voxels containing point data points, the point cloud intensity value can be determined accordingly. For example, if the voxel contains at least one point, the point cloud intensity value corresponding to each point within the voxel can be directly determined. Alternatively, the point cloud intensity value within the voxel can be determined to include a combined intensity value. This combined intensity value can be determined based on point cloud intensity information from multiple dimensions within the voxel. For example, the point cloud intensity information from multiple dimensions within the voxel includes the average, minimum, and maximum point cloud intensity values ​​within the voxel, and the combined intensity value is formed by at least two combinations of these values. For voxels that do not contain point data points, the point cloud intensity value within that voxel is zero.

[0050] The input matrix, formed by combining the point cloud coordinates and intensity values ​​within each voxel, refers to an N×4 input matrix, where N represents the number of points in the point cloud data. Each row of the input matrix contains four values: "x, y, z, g", where "x, y, z" represent the point coordinates, and "g" represents the point cloud intensity value within the corresponding voxel. As mentioned earlier, the point cloud intensity value within a voxel can be the intensity value of each point within the voxel, or it can be a combined intensity value. When it is a combined intensity value, it can increase the receptive field during convolution processing, leading to more accurate object recognition.

[0051] In some embodiments, for a voxel at position [i,j,k] in space, the coordinate values ​​of each point within the voxel need to undergo voxel transformation. For example, if the coordinate values ​​of each point are represented as (x,y,z), the transformation formula is "(x,y,z)-voxel_min_range / voxel_ratio", where voxel_min_range represents the minimum voxel boundary position, and voxel_ratio represents the voxel resolution. The voxel-transformed point coordinate values ​​can form a data input matrix to increase the correlation between features and voxels within the data input matrix, thereby improving the accuracy of object recognition.

[0052] In step S230, the process of convolution processing on the input matrix is ​​also the process of feature extraction on the input matrix, which can be expressed as the formula: F = Conv2D(reduce_dims(Conv3D(input))). This can be understood as first increasing the dimensionality of the feature data contained in the input matrix through three-dimensional convolution processing to obtain a three-dimensional object feature matrix, and then performing two-dimensional convolution processing on the three-dimensional object feature matrix to obtain a classification matrix and a regression matrix.

[0053] For example, the 3D object feature matrix obtained by performing 3D convolution on the input matrix can be represented as a five-dimensional feature vector matrix [b,c,d,h,w], where b represents the bit-sized parameter of the network model, c represents the dimension of the point cloud intensity value, d represents the spatial range of the corresponding voxel, and h and w represent the feature location. In some embodiments, to reduce dimensionality and subsequent computation, dimensions c and d can be concatenated to obtain a four-dimensional feature vector matrix [b,c*d,h,w], which is then subjected to 2D convolution.

[0054] The two-dimensional convolution processing proposed in this embodiment requires the output of two feature vector matrices: a classification matrix and a regression position matrix. The classification matrix characterizes whether there are object position points within each voxel, while the regression position matrix includes the center point position within the voxel and the offset of the object position point within the voxel relative to the center point position. Here, an object position point refers to a point used to identify the position of an object, particularly a key position point after the object has been abstracted in one or two dimensions. For example, if the object is a rod-shaped object, the object position points include the top and bottom vertices after the rod-shaped object has been abstracted into a one-dimensional line segment; if the object is a planar sign-shaped object, the object position points include the vertices of each edge after the planar sign-shaped object has been abstracted into a two-dimensional polygon. Therefore, it can be understood that compared to the three-dimensional bounding box of the object predicted by existing technologies, the object position points involved in this embodiment are points that can more easily identify the characteristics of the object. This allows the corresponding object to be identified in step S240 based on the position coordinates of each object position point within the voxel, achieving a simpler object identification. Furthermore, the amount of annotation work required for the object location points in the training data involved in this embodiment is significantly less than the amount of annotation work required in the prior art when recognizing 3D bounding boxes, which can obtain more training data more efficiently and at a lower cost.

[0055] For example, in the process of two-dimensional convolution processing, a fully connected layer can be used to output a classification matrix after the convolution operation in the direction of the object's center line, and a regression position matrix can be output after the convolution operation in the direction of the object's vertical line.

[0056] If we represent the classification matrix as S and the regression position matrix as P, then we have the following formula:

[0057] S∈R mhw , P∈R mnhw

[0058] Where R represents the 3D object feature matrix obtained after the input matrix is ​​processed by 3D convolution, m corresponds to the m objects identified, that is, any feature position S(h,w) on the classification matrix S outputs a vector of length m, and n represents the parameters in the regression process.

[0059] It should be noted that this embodiment can use deep learning network models such as UNet and CNN to perform convolution processing on the input matrix, but does not limit the specific model structure.

[0060] In step S240, since the classification matrix represents whether there is an object location point in each voxel, and the regression position matrix contains the center point position in the voxel and the offset of the object location point in the voxel relative to the center point position, the position coordinates of the object location points in the voxel can be determined according to the classification matrix and the regression position matrix, and then the corresponding object can be identified according to the position coordinates of these object location points.

[0061] For example, since any feature position S(h,w) on the classification matrix S outputs a vector of length m, corresponding to m objects, if it is determined that there is an object location point at a certain feature position based on the output vector of that feature position, then the regression position information is obtained at the corresponding feature position in the regression position matrix P, thereby calculating the position coordinates of the corresponding object location point. Specifically, this may include the following process:

[0062] Traverse the classification location matrix to determine the feature locations where the classification value is greater than a preset threshold;

[0063] The regression position information of the corresponding feature position is determined from the regression position matrix. The regression position information includes the position of the center point within the corresponding voxel and the offset of each object position point within the corresponding voxel relative to the center point position.

[0064] Calculate the position coordinates of each object point within the voxel based on the determined center point position and offset.

[0065] It should be noted that feature locations with classification values ​​greater than a preset threshold indicate that these feature locations are object locations where the object to be identified exists. In some embodiments, the position coordinates of the corresponding object location point can be obtained by adding the center point position and the offset. In other embodiments, for example, when the point cloud coordinates of each voxel in the input matrix are obtained after voxel transformation of the coordinates of each point within the voxel, the initial position information is obtained by calculating the sum of the center point position and the offset. Then, the position coordinates of each object location point within the corresponding voxel are obtained by calculating the product of the initial position information and the voxel resolution, and the sum of the calculated product value and the minimum boundary position information of the corresponding voxel. This calculation process can be expressed as the following formula:

[0066] positions=regs[h,w]+offs[h,w]

[0067] poles=positions*voxel_ratio+voxel_min_range

[0068] Where positions represent initial position information, regs[h,w] represents center point position, offs[h,w] represents offset, polls represents position coordinates, voxel_ratio represents voxel resolution, and voxel_min_range represents voxel minimum boundary position.

[0069] As can be seen from the above, this embodiment obtains multiple voxels within the point cloud range and the point cloud intensity value corresponding to each voxel by spatial rasterization of the point cloud data. Then, it performs convolution processing on the input matrix formed by combining the point cloud coordinate values ​​and point cloud intensity values ​​within each voxel to obtain the classification matrix and regression position matrix respectively. Finally, it determines the position coordinates of each object position point within each voxel based on these two matrices and identifies the object by using the position coordinates of each object position point within the voxel. It is evident that the object recognition in this embodiment is achieved based on the position coordinates of the object position points within each voxel. Compared with the prior art, it does not require the identification and annotation of the object's 3D bounding box. Therefore, the object recognition scheme proposed in this embodiment has higher recognition simplicity and efficiency when recognizing targets such as poles and signs, as it does not require the identification of the target's 3D bounding box as when recognizing other targets such as vehicles.

[0070] Please see Figure 3 , Figure 3 Is Figure 2 A flowchart of another object recognition method proposed based on the illustrated embodiment. For example... Figure 3 As shown, the method further includes step S310 before step S210: converting the coordinate information contained in the point cloud data into coordinate information in the vehicle coordinate system, so as to perform spatial rasterization processing based on the converted point cloud data; and step S240 further includes: converting the position coordinates of each object position point in the voxel into coordinate information in the geocentric coordinate system, and identifying the corresponding object through the coordinate information in the geocentric coordinate system.

[0071] In this embodiment, the point cloud data is obtained by the vehicle scanning the objects to be identified on the traffic road using LiDAR signals, and the point cloud coordinate system is generally a geocentric coordinate system. Considering that there is a certain deviation between the point cloud coordinate system and the vehicle coordinate system, in step S310, the coordinate information contained in the point cloud data is converted into coordinate information in the vehicle coordinate system, so that subsequent spatial rasterization processing can be performed based on the converted point cloud data, thereby improving the accuracy of object recognition to a certain extent.

[0072] For example, the direction and position of each frame of point cloud in the vehicle coordinate system can be determined based on the vehicle's driving trajectory information. The transfer matrix can be calculated based on the direction and position of the point cloud in the vehicle coordinate system. Based on the transfer matrix, the coordinate information contained in the point cloud data can be converted into coordinate information in the vehicle coordinate system.

[0073] Therefore, the position coordinates of the object location points identified in step S230 should also be the position information in the vehicle coordinate system. Thus, by converting the position coordinates of each object location point in the voxel into coordinate information in the geocentric coordinate system, and using the coordinate information in the geocentric coordinate system to identify the corresponding object, the position coordinates of the identified object can be restored to the point cloud coordinate system. This also facilitates the subsequent application of the solution proposed in this embodiment to intelligent transportation programs such as map production and intelligent driving.

[0074] In another exemplary embodiment, Figure 2 and Figure 3 Steps S230 and S240 in the illustrated object recognition method are implemented by a pre-trained object recognition model, which refers to a machine learning model. That is, in this embodiment, the pre-trained object recognition model performs convolution processing on the input matrix to obtain a classification matrix and a regression position matrix. Based on the classification and regression position matrices, it determines the position coordinates of the object's location points within the voxel, and identifies the corresponding object using these coordinates. By inputting the input matrix into the object recognition model, the object's position output by the model can be obtained.

[0075] In this embodiment, the object recognition method further includes a process of training an object recognition model. In an exemplary embodiment, the process of training the object recognition model may include, for example... Figure 4 The processing flow shown, or including more Figure 4 The number of processing steps can be adjusted according to actual needs. Additionally, Figure 4 The processing flows shown have been described in the foregoing embodiments, and detailed processing flows can be found in the descriptions of the foregoing embodiments. It should also be understood that the object recognition model also executes the steps shown in S230 and S240 during the training process.

[0076] like Figure 5 As shown, in an exemplary embodiment, the process of training the object recognition model includes the following steps S510 to S530:

[0077] S510, during the process of training the object recognition model using point cloud training data containing coordinate information of the objects to be recognized and position annotation data of the objects to be recognized, obtains the position recognition result of the objects to be recognized output by the object recognition model.

[0078] S520, calculate a loss value of at least one dimension based on the position recognition result of the trained object and the position annotation data of the trained object, wherein the at least one dimension includes at least one of the following: classification dimension, position regression dimension, and center point compensation dimension.

[0079] S530, determine the training loss value based on the loss value of at least one dimension, so as to train the object recognition model using the training loss value.

[0080] It should be noted that in step S510, the position of the object to be recognized in the point cloud data is known; its specific position can be found in the corresponding location annotation data. The training process of the object recognition model involves using point cloud training data containing the coordinate information of the object to be recognized, as well as the location annotation data of the object to be recognized, to train the object recognition model. During the training process, the input matrix to the object recognition model is also processed by spatial rasterization of the point cloud training data as in steps S510 and S520, obtaining multiple voxels within the corresponding point cloud coordinate range, and determining the point cloud intensity value within each voxel. This intensity value is formed by combining the corresponding point cloud coordinate value and point cloud intensity value within each voxel. The object recognition model outputs the position recognition result of the object to be recognized based on this input matrix.

[0081] It should be noted that during the training process of the object recognition model, the positions of the training objects can be labeled according to the type of object being recognized. For example, for the recognition of pole-shaped objects, such as streetlights and traffic lights, pole-shaped objects are selected as the training objects, and the position labeling data of the training objects is specifically the line segments formed by connecting the top and bottom vertices of the training objects. Training the object recognition model with this type of training data can result in a better performance in pole-shaped object recognition scenarios. For example... Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the effect of object recognition using an object recognition model in an exemplary application scenario: the recognition of rod-shaped objects. Figure 6 The left image shows the point cloud collected by the vehicle during its journey. The input matrix is ​​determined based on the corresponding point cloud data and then input into the trained object recognition model. The rod-shaped objects identified by the object recognition model are shown in the right image. It can be seen that the rod-shaped objects identified by the object recognition model are marked using abstract one-dimensional line segments.

[0082] For example, in the recognition of planar objects such as traffic signs and billboards, planar objects are used for training. The positional annotation data of the training objects are specifically the shapes formed by connecting the vertices of each side of the abstracted two-dimensional polygon, such as rectangles and triangles. Training the object recognition model with this type of training data can result in better performance in planar object recognition scenarios. Therefore, the selection of actual training data can be determined according to the actual application scenario, and this section does not impose any restrictions on it.

[0083] It should also be noted that, as can be seen from the above, compared with the object annotation based on 3D bounding boxes in the existing technology, the training process of the object recognition model provided in this embodiment supports a simpler sample annotation method, that is, only one-dimensional or two-dimensional position information annotation is required. This not only can a more efficient object recognition model be obtained, but also the training cost of the model is reduced, which can make the object recognition model have a wider range of applications.

[0084] In step S520, a loss value of at least one dimension can be calculated based on the position recognition results of the trained object and the position annotation data of the trained object. The at least one dimension includes at least one of the following: classification dimension, position regression dimension, and center point compensation dimension. Different dimensions represent the loss of the object recognition model at different levels.

[0085] In step S530, if the loss value calculated in step S520 includes three dimensions: classification dimension, location regression dimension, and center point compensation dimension, then the training loss value can be calculated using the following formula:

[0086] loss=cls_loss*weight1+reg_loss*weight2+off_loss*weight3

[0087] Where cls_loss represents the loss value of the classification dimension, weight1 represents the weight of the classification dimension, reg_loss represents the loss value of the location regression dimension, weight2 represents the weight of the location regression dimension, off_loss represents the loss value of the center point compensation dimension, and weight3 represents the weight of the center point compensation dimension.

[0088] As can be seen from the above, the training loss value is obtained by weighting and calculating the loss values ​​of multiple dimensions. This allows the object recognition model proposed in this exemplary embodiment to consider loss information of different dimensions during the training process. The trained object recognition model can have better model performance, that is, it has more accurate object recognition performance.

[0089] In another exemplary embodiment, such as Figure 7As shown, the process of training the object recognition model includes the following steps S710 to S730:

[0090] S710, in the process of training the object recognition model using point cloud training data containing coordinate information of the objects to be recognized and position annotation data of the objects to be recognized, obtains the initial training results and calculates the initial loss value.

[0091] S720 determines the initial recognition position of the training object based on the initial training results, and performs re-recognition processing on the surrounding point cloud data with the initial recognition position as the center to obtain the re-recognition result of the training object's position, and calculates the re-loss value. In the re-recognition process, the spatial rasterization processing of the surrounding point cloud data is performed again with the initial recognition position of the training object.

[0092] S730, calculate the weighted sum of the initial loss value and the subsequent loss value to obtain the training loss value, so as to train the object recognition model using the training loss value.

[0093] In this embodiment, the process of training the object recognition model based on point cloud training data containing coordinate information of the objects to be recognized and the position annotation data of the objects to be recognized can be summarized as including two processes: "coarse extraction" and "fine detection".

[0094] The “coarse extraction” process is shown in step S710. The object recognition model is trained using the above training data to obtain the initial training results. The initial training results include the initial recognition position of the trained object. The initial loss value can be calculated based on the initial training results and the position annotation data of the trained object. This will not be elaborated on further here.

[0095] If the "coarse extraction" process can be summarized as the initial object recognition process for the training object, then the "fine detection" process can be correspondingly summarized as the re-recognition process for the training object. The re-recognition process, as shown in step S720, involves re-processing the surrounding point cloud data around the initial recognition position of the training object to obtain the re-recognition result of the training object's position. The process of re-rasterizing the surrounding point cloud data based on the initial recognition position of the training object can be understood as determining the initial position information of the training object based on the initial recognition result. Further recognizing the position of the training object based on this initial position information can improve the recognition accuracy of the finally trained object recognition model.

[0096] For example, if the object being trained to be recognized is a rod-shaped object, the rasterization of the surrounding point cloud data is performed again based on the initial recognition position of the object being trained. This includes, for example, transforming the point cloud coordinate system to use the rod of the object being trained as the z-axis and the opposite direction of the vehicle trajectory as the x-axis, thereby further determining the y-axis to obtain a point cloud coordinate system that is more consistent with the actual scene, resulting in more accurate point cloud coordinate information. The accuracy of the object recognition model in recognizing the object is also improved accordingly. Alternatively, the voxel resolution can be adjusted according to the initial recognition position of the object being trained, also for the purpose of improving the accuracy of the object recognition model in recognizing the object. This is not a limitation.

[0097] It should be noted that, apart from the different process of re-performing spatial rasterization, the operation of the object recognition model is the same as the initial object recognition process during the re-recognition process.

[0098] In step S730, a training loss value is obtained by calculating the weighted sum of the initial loss value and the subsequent loss value, and the object recognition model is trained using the training loss value. The calculation process of the training loss value can be expressed as the following formula:

[0099] loss(final)=loss(step1)*weight(step1)+loss(step2)*weight(step2)

[0100] Where loss(final) represents the training loss value, loss(step1) represents the initial loss value, weight(step1) represents the weight corresponding to the initial loss value, loss(step2) represents the second loss value, and weight(step2) represents the weight corresponding to the second loss value.

[0101] As can be seen from the above, the training loss value is obtained by weighting and calculating the initial loss value and the subsequent loss value. This allows the object recognition model proposed in this exemplary embodiment to determine the initial position information of the object to be recognized based on the preliminary recognition results during the training process. Based on this initial position information, the position of the object to be recognized can be further identified. This training method can improve the recognition accuracy of the finally trained object recognition model.

[0102] It should be noted that in practical application scenarios, Figure 5 and Figure 7 The training methods for the object recognition model shown can be executed separately or in combination to significantly improve the accuracy of the trained object recognition model. This can be determined according to actual needs.

[0103] Figure 8This is a block diagram illustrating an object recognition device according to an exemplary embodiment of this application. The device is suitable for… Figure 1 The implementation environment shown is, for example, configured in Figure 1 The device is used on the vehicle-mounted terminal 110 or vehicle-mounted server 120 in the illustrated implementation environment. Of course, this device can also be applied to other implementation environments and configured on terminal devices or server devices in other implementation environments to achieve object recognition based on point cloud data. This embodiment does not limit this.

[0104] like Figure 8 As shown, in an exemplary embodiment, the device includes:

[0105] The rasterization processing module 810 is configured to perform spatial rasterization processing on point cloud data to obtain multiple voxels within the point cloud coordinate range; the input matrix determination module 820 is configured to determine the point cloud intensity value within each voxel, and combine the point cloud coordinate value and the point cloud intensity value within each voxel to form an input matrix; the convolution processing module 830 is configured to perform convolution processing on the input matrix to obtain a classification matrix and a regression position matrix, wherein the classification matrix represents whether there is an object position point within each voxel, and the regression position matrix contains the center point position within the voxel and the offset of the object position point within the voxel relative to the center point position; the object recognition module 840 is configured to determine the position coordinates of the object position points within the voxels according to the classification position matrix and the regression position matrix, and identify the corresponding object by the position coordinates of the object position points within the voxels.

[0106] This device obtains multiple voxels and their corresponding point cloud intensity values ​​within the point cloud range by spatially rasterizing the point cloud data. Then, it performs convolution processing on the input matrix formed by combining the point cloud coordinates and intensity values ​​within each voxel to obtain a classification matrix and a regression position matrix. Finally, it determines the position coordinates of each object point within each voxel based on these two matrices and identifies the object using these position coordinates. It is evident that this device identifies objects based on the position coordinates of object points within each voxel, without needing to identify and label the object's 3D bounding box. Therefore, the object recognition device proposed in this embodiment has higher object recognition simplicity and improves object recognition efficiency.

[0107] In another exemplary embodiment, the point cloud intensity values ​​within each voxel include combined intensity values; the input matrix determination module 820 includes:

[0108] The combined intensity value determination unit is configured to determine the combined intensity value corresponding to each voxel based on the point cloud intensity information of multiple dimensions within each voxel; the combined intensity value processing unit is configured to combine the point cloud coordinate values ​​and point cloud intensity values ​​within each voxel to form an input matrix.

[0109] In another exemplary embodiment, the combined intensity value determination unit is configured to: calculate the average, minimum, and maximum value of the point cloud intensity values ​​within each voxel; and form a combined intensity value for the corresponding voxel from at least two combinations of the average, minimum, and maximum values.

[0110] In another exemplary embodiment, the object recognition module includes:

[0111] The classification matrix traversal unit is configured to traverse the classification position matrix to determine the feature positions where the classification value is greater than a preset threshold; the regression position determination unit is configured to determine the regression position information of the corresponding feature position from the regression position matrix, the regression position information including the center point position within the corresponding voxel and the offset of each object position point within the corresponding voxel relative to the center point position; the position coordinate calculation unit is configured to calculate the position coordinates of each object position point within the voxel based on the determined center point position and offset.

[0112] In another exemplary embodiment, the point cloud coordinates of each voxel in the input matrix are obtained by voxel transformation of the coordinates of each point in the voxel; the position coordinate calculation unit is configured to: calculate the sum of the center point position and the offset to obtain the initial position information; calculate the product of the initial position information and the voxel resolution, and the sum of the calculated product value and the corresponding voxel minimum boundary position to obtain the position coordinates of each object position point in the corresponding voxel.

[0113] In another exemplary embodiment, the convolution processing module is configured as follows:

[0114] The input matrix is ​​subjected to 3D convolution to obtain a 3D object feature matrix; the 3D object feature matrix is ​​then subjected to 2D convolution to obtain a classification matrix and a regression position matrix.

[0115] In another exemplary embodiment, the point cloud data is obtained by the vehicle scanning objects to be identified on the traffic road using LiDAR signals; the device further includes: a coordinate system transformation module configured to convert the coordinate information contained in the point cloud data into coordinate information in the vehicle coordinate system, so as to perform spatial rasterization processing based on the converted point cloud data; and an object recognition module configured to convert the position coordinates of each object location point in the voxel into coordinate information in the geocentric coordinate system, and identify the corresponding object through the coordinate information in the geocentric coordinate system.

[0116] In another exemplary embodiment, the device further includes a first model training module configured as follows:

[0117] In the process of training an object recognition model using point cloud training data containing coordinate information of the objects to be recognized and position annotation data of the objects to be recognized, the position recognition result of the objects to be recognized output by the object recognition model is obtained; a loss value of at least one dimension is calculated based on the position recognition result and the position annotation data of the objects to be recognized, the at least one dimension including at least one of classification dimension, position regression dimension, and center point compensation dimension; a training loss value is determined based on the loss value of at least one dimension, so as to train the object recognition model through the training loss value.

[0118] In another exemplary embodiment, the device further includes a second model training module configured as follows:

[0119] In the process of training an object recognition model using point cloud training data containing the coordinate information of the objects to be recognized and the location annotation data of the objects to be recognized, the initial training results are obtained and the initial loss value is calculated. Based on the initial training results, the initial recognition position of the objects to be recognized is determined, and the surrounding point cloud data is re-recognized with the initial recognition position as the center to obtain the re-recognition result of the position of the objects to be recognized, and the re-loss value is calculated. In the re-recognition process, the surrounding point cloud data is re-spatialized with the initial recognition position of the objects to be recognized. The weighted sum of the initial loss value and the re-loss value is calculated to obtain the training loss value, which is used to train the object recognition model.

[0120] It should be noted that the apparatus and method provided in the above embodiments belong to the same concept, and the specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the apparatus provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation.

[0121] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the object recognition methods provided in the above embodiments.

[0122] Figure 9 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 9 The computer system of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0123] like Figure 9 As shown, the computer system includes a Central Processing Unit (CPU) 901, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 902 or programs loaded from storage portion 908 into Random Access Memory (RAM) 903, such as performing the methods described in the above embodiments. The RAM 903 also stores various programs and data required for system operation. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An Input / Output (I / O) interface 905 is also connected to the bus 904.

[0124] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. Removable media 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 99 as needed so that computer programs read from them can be installed into storage section 908 as needed.

[0125] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs various functions defined in the system of this application.

[0126] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0129] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the object recognition method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0130] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the object recognition methods provided in the various embodiments described above.

[0131] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. An object recognition method, characterized in that, The method includes: Spatial rasterization is performed on the point cloud data to obtain multiple voxels within the point cloud coordinate range; Combine at least two of the average, minimum, and maximum point cloud intensity values ​​within each voxel to obtain a combined intensity value within that voxel, and combine the point cloud coordinate values ​​within each voxel with the combined intensity value to form an input matrix; The input matrix is ​​subjected to three-dimensional convolution to obtain a three-dimensional object feature matrix. The three-dimensional object feature matrix is ​​then subjected to two-dimensional convolution in the direction of the object's center line and the direction perpendicular to the object to obtain a classification matrix and a regression position matrix. The classification matrix represents whether there is an object position point in each voxel, and the object position point is a key position point used to identify the object after one-dimensional or two-dimensional abstraction. The regression position matrix includes the center point position in the voxel and the offset of the object position point in the voxel relative to the center point position. The position coordinates of the object's location points within the voxel are determined based on the classification matrix and the regression position matrix, and the corresponding object is identified by the position coordinates of the object's location points within the voxel.

2. The method according to claim 1, characterized in that, Determining the position coordinates of an object point within a voxel based on the classification matrix and the regression position matrix includes: Traverse the classification matrix to determine the feature locations where the classification value is greater than a preset threshold; The regression position information of the corresponding feature position is determined from the regression position matrix. The regression position information includes the position of the center point within the corresponding voxel and the offset of each object position point within the corresponding voxel relative to the center point position. The position coordinates of each object point within the voxel are calculated based on the determined center point position and the offset.

3. The method according to claim 2, characterized in that, The point cloud coordinates of each point in the input matrix are obtained by performing voxel transformation on the coordinates of each point in the voxel. The step of calculating the position coordinates of each object point within the voxel based on the determined center point position and the offset includes: Calculate the sum of the center point position and the offset to obtain the initial position information; The product of the initial position information and the voxel resolution is calculated, and the sum of the calculated product value and the corresponding minimum boundary position of the voxel is obtained to obtain the position coordinates of each object position point within the corresponding voxel.

4. The method according to claim 1, characterized in that, The step of obtaining the classification matrix and regression position matrix by performing convolution processing on the input matrix includes: The input matrix is ​​subjected to 3D convolution processing to obtain a 3D object feature matrix; The feature matrix of the three-dimensional object is subjected to two-dimensional convolution to obtain the classification matrix and the regression position matrix.

5. The method according to claim 1, characterized in that, The point cloud data is obtained by a vehicle scanning objects to be identified on the road using lidar signals; before spatial rasterization processing of the point cloud data, the method further includes: The coordinate information contained in the point cloud data is converted into coordinate information in the vehicle coordinate system, so as to perform the spatial rasterization process based on the converted point cloud data. The step of identifying the corresponding object by the position coordinates of the object's location point within the voxel includes: The position coordinates of each object point within the voxel are converted into coordinate information in the geocentric coordinate system, and the corresponding object is identified by the coordinate information in the geocentric coordinate system.

6. The method according to any one of claims 1-5, characterized in that, The steps of obtaining a classification matrix and a regression position matrix by performing convolution processing on the input matrix, and the steps of determining the position coordinates of the object position points within the voxel based on the classification matrix and the regression position matrix, and identifying the corresponding object based on the position coordinates of the object position points within the voxel, are executed by a pre-trained object recognition model. The method further includes: During the process of training the object recognition model using point cloud training data containing coordinate information of the trained objects and position annotation data of the trained objects, the position recognition result of the trained objects output by the object recognition model is obtained. Based on the position recognition results of the trained objects and the position annotation data of the trained objects, a loss value of at least one dimension is calculated, wherein the at least one dimension includes at least one of the following: classification dimension, position regression dimension, and center point compensation dimension. A training loss value is determined based on the loss value of at least one dimension, so as to train the object recognition model using the training loss value.

7. The method according to any one of claims 1-5, characterized in that, The steps of obtaining a classification matrix and a regression position matrix by performing convolution processing on the input matrix, and the steps of determining the position coordinates of the object position points within the voxel based on the classification matrix and the regression position matrix, and identifying the corresponding object based on the position coordinates of the object position points within the voxel, are executed by a pre-trained object recognition model. The method further includes: During the process of training the object recognition model using point cloud training data containing coordinate information of the objects to be recognized and the position annotation data of the objects to be recognized, the initial training results are obtained and the initial loss value is calculated. The initial recognition position of the training object is determined based on the initial training results, and the surrounding point cloud data is re-recognized with the initial recognition position as the center to obtain the re-recognition result of the training object's position, and the re-loss value is calculated. In the re-recognition process, the spatial rasterization of the surrounding point cloud data is re-processed with the initial recognition position of the training object. The initial loss value and the subsequent loss value are weighted and summed to obtain the training loss value, which is then used to train the object recognition model.

8. An object recognition device, characterized in that, The device includes: The rasterization processing module is configured to perform spatial rasterization processing on point cloud data to obtain multiple voxels within the point cloud coordinate range; The input matrix determination module is configured to combine at least two of the average, minimum and maximum values ​​of the point cloud intensity values ​​in each voxel to obtain the combined intensity value in that voxel, and to form an input matrix by combining the point cloud coordinate values ​​in each voxel and the combined intensity value. The convolution processing module is configured to perform three-dimensional convolution processing on the input matrix to obtain a three-dimensional object feature matrix, and to perform two-dimensional convolution processing on the three-dimensional object feature matrix in the direction of the object centerline and the direction of the object perpendicular to the line to obtain a classification matrix and a regression position matrix. The classification matrix represents whether there is an object position point in each voxel, and the object position point is a key position point used to identify the object after one-dimensional or two-dimensional abstraction. The regression position matrix includes the center point position in the voxel and the offset of the object position point in the voxel relative to the center point position. The object recognition module is configured to determine the position coordinates of the object location points within the voxel based on the classification matrix and the regression position matrix, and to identify the corresponding object by using the position coordinates of the object location points within the voxel.

9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the method of any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-7.

Images