A target identification method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a target identification method and device, electronic equipment and storage medium, and relate to the technical field of intelligent transportation. The method comprises: acquiring a radar point cloud image collected by a radar; based on a pre-trained time domain convolutional neural network and a frequency domain convolutional neural network, respectively performing target detection on the radar point cloud image to obtain a first target list and a second target list of the radar point cloud image, wherein the first target list and the second target list each include at least one target identification result; and matching the first target list and the second target list to obtain a target identification result of the radar point cloud image. The embodiments of the present application improve the accuracy and efficiency of target identification.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a target recognition method, device, electronic device, and storage medium. Background Technology

[0002] In the field of intelligent transportation technology, using LiDAR (Light Detection and Ranging) to capture point cloud images and then identifying and classifying targets based on these images is a fundamental function of the LiDAR system required for future autonomous vehicles. This helps reduce the data transmission latency between the LiDAR system and the central processing unit in autonomous vehicles, further reducing the processing time for target identification. This allows the central processing unit to have a longer response time to combine information from all types of sensors to make correct decisions and effectively avoid traffic accidents.

[0003] In existing technologies, convolutional neural network models based on the time domain are often used to identify target features. However, when there are multiple targets in a point cloud image, these targets may have irregular, complex, fractal geometric shapes or attribute information that is difficult to extract by time domain detection, resulting in low accuracy and efficiency in target identification. Summary of the Invention

[0004] The purpose of this invention is to provide a target recognition method, apparatus, electronic device, and storage medium to improve the accuracy and efficiency of target recognition. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a target recognition method, the method comprising:

[0006] Acquire radar point cloud images collected by radar;

[0007] Based on pre-trained temporal convolutional neural networks and frequency convolutional neural networks, target detection is performed on the radar point cloud image to obtain a first target list and a second target list of the radar point cloud image, wherein the first target list and the second target list each include the recognition result of at least one target.

[0008] The first target list and the second target list are matched to obtain the target recognition result of the radar point cloud image.

[0009] In one embodiment of the present invention, matching the first target list and the second target list to obtain the target recognition result of the radar point cloud image includes:

[0010] For any identification result in the first target list, if there is a matching identification result in the second target list, that identification result shall be taken as the target identification result.

[0011] In one embodiment of the present invention, matching the first target list and the second target list to obtain the target recognition result of the radar point cloud image includes:

[0012] When there are mismatched identification results in the first target list and the second target list, time-frequency analysis is performed on the target to be processed corresponding to the mismatched identification result to determine the target identification result.

[0013] In one embodiment of the present invention, the identification result includes: classification information of the target corresponding to the identification result; the step of performing time-frequency analysis on the target to be processed corresponding to the mismatched identification result to determine the target identification result includes:

[0014] Acquire a local image of the target to be processed within a preset range in the radar point cloud image;

[0015] The time-frequency spectrum of the local image is obtained, and the environmental information of the target to be processed is obtained by analyzing the time-frequency spectrum.

[0016] The attribute information of the target to be processed is determined based on the time-frequency spectrum, wherein the attribute information includes geometric shape;

[0017] The classification information of the target to be processed is determined based on the attribute information;

[0018] If the environmental information and the classification information meet the preset matching principle, the classification information is used as the target identification result.

[0019] In one embodiment of the present invention, the environmental information includes environmental objects;

[0020] The step of using the classification information as the target recognition result when the environmental information and the classification information meet a preset matching principle further includes:

[0021] Determine whether the relative distance between the target to be processed and the environmental object meets a preset distance standard. If so, determine that the environmental information and the classification information conform to a preset matching principle. The environmental object includes road signs and / or at least one target in the first target list and / or the second target list other than the target to be processed.

[0022] Secondly, embodiments of the present invention also provide a target recognition device, the device comprising:

[0023] The image acquisition module is used to acquire radar point cloud images collected by the radar.

[0024] The target list acquisition module is used to perform target detection on the radar point cloud image based on a pre-trained temporal convolutional neural network and a frequency convolutional neural network, respectively, to obtain a first target list and a second target list of the radar point cloud image, wherein the first target list and the second target list each include the recognition result of at least one target;

[0025] The identification result determination module is used to match the first target list and the second target list to obtain the target identification result of the radar point cloud image.

[0026] In one embodiment of the present invention, the identification result determination module is specifically used for:

[0027] For any identification result in the first target list, if there is a matching identification result in the second target list, that identification result shall be taken as the target identification result.

[0028] In one embodiment of the present invention, the identification result determination module is specifically used for:

[0029] When there are mismatched identification results in the first target list and the second target list, time-frequency analysis is performed on the target to be processed corresponding to the mismatched identification result to determine the target identification result.

[0030] In one embodiment of the present invention, the identification result includes: classification information of the target corresponding to the identification result; the identification result determination module is specifically used for:

[0031] Acquire a local image of the target to be processed within a preset range in the radar point cloud image;

[0032] The time-frequency spectrum of the local image is obtained, and the environmental information of the target to be processed is obtained by analyzing the time-frequency spectrum.

[0033] The attribute information of the target to be processed is determined based on the time-frequency spectrum, wherein the attribute information includes geometric shape;

[0034] The classification information of the target to be processed is determined based on the attribute information;

[0035] If the environmental information and the classification information meet a preset matching principle, the classification information is used as the target identification result;

[0036] In one embodiment of the present invention, the environmental information includes environmental objects; the identification result determination module is specifically used for:

[0037] Determine whether the relative distance between the target to be processed and the environmental object meets a preset distance standard. If so, determine that the environmental information and the classification information conform to a preset matching principle. The environmental object includes road signs and / or at least one target in the first target list and / or the second target list other than the target to be processed.

[0038] Thirdly, embodiments of the present invention also provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0039] Memory, used to store computer programs;

[0040] When a processor executes a program stored in memory, it implements any of the target recognition method steps described above.

[0041] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the target recognition method steps described above.

[0042] Fifthly, embodiments of the present invention also provide a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the target recognition methods described above.

[0043] Beneficial effects of the embodiments of the present invention:

[0044] The target recognition method provided in this invention first acquires a radar point cloud image collected by radar. Then, based on pre-trained temporal convolutional neural networks and frequency-domain convolutional neural networks, target detection is performed on the radar point cloud image to obtain a first target list and a second target list. Each target list includes the recognition result of at least one target. The first and second target lists are then matched to obtain the target recognition result of the radar point cloud image. This invention performs target detection on the radar point cloud image based on both temporal and frequency-domain convolutional neural networks, and combines the recognition results from both domains for comprehensive judgment. This approach achieves more efficient and accurate target recognition results, better utilizing the features of different types of convolutional neural networks to assist in target recognition in the point cloud image, thereby improving the accuracy and efficiency of target recognition.

[0045] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0047] Figure 1 A flowchart illustrating the first target recognition method provided in an embodiment of the present invention;

[0048] Figure 2 This is a flowchart illustrating the second target recognition method provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the structure of a target recognition device provided in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0052] Since the accuracy and efficiency of target recognition in existing technologies for radar point cloud images are relatively low, in order to solve this problem, embodiments of the present invention provide a target recognition method, device, electronic device and storage medium. The target recognition method will be described in detail below through specific embodiments.

[0053] The method of this invention is applied to a smart terminal and can be implemented through a smart terminal. In actual use, the smart terminal can be an in-vehicle intelligent system, an in-vehicle data processing center, a cloud server, etc.

[0054] See Figure 1 The present invention provides a flowchart of a first target recognition method, including:

[0055] Step S11: Acquire radar point cloud images collected by radar.

[0056] The radar used to acquire the radar point cloud images mentioned above can be a lidar. A point cloud refers to a collection of multiple points that can represent an object, and it can represent the object's three-dimensional coordinates, spatial outline, color information, reflection intensity information, echo count, etc.

[0057] Step S12: Based on the pre-trained temporal convolutional neural network and frequency convolutional neural network, target detection is performed on the radar point cloud image to obtain the first target list and the second target list of the radar point cloud image.

[0058] The first target list and the second target list each include the identification results of at least one target.

[0059] Convolutional neural networks (CNNs) can be used to extract features from images, thereby enabling target detection. The time domain describes the relationship between mathematical functions or physical signals and time; the frequency domain describes the frequency characteristics of a signal. In this embodiment of the invention, pre-trained time-domain and frequency-domain CNNs are simultaneously used to extract features from the aforementioned radar point cloud image. Considering the mutual influence between the signal and the interconnects, target detection in the radar point cloud image is achieved.

[0060] The pre-training of the aforementioned temporal and frequency-domain convolutional neural networks can be achieved using publicly available image databases. The temporal convolutional neural network is trained by calculating coefficients related to parallel linear combination equations, ultimately yielding coefficients for different target classifications corresponding to different linear combination equations. The frequency-domain neural network, on the other hand, requires first performing a two-dimensional Fourier transform on each image in the relevant image database to obtain Fourier coefficients for each image, and then calculating different coefficients for different target classifications based on these Fourier coefficients. After the temporal and frequency-domain convolutional neural networks are trained until convergence, target detection is performed on radar point cloud images based on the final obtained coefficients.

[0061] Specifically, the structures of temporal convolutional neural networks and frequency convolutional neural networks can be referenced from network structures such as R-CNN, ConvNet, Fast R-CNN, Mask R-CNN, and Multitask R-CNN (the above English names are all types of convolutional neural networks).

[0062] The aforementioned targets refer to those that need to be detected in this application scenario, such as pedestrians, animals and plants, vehicles, roadblocks, and signs. In practical applications, radar point cloud images contain at least one target. Target detection is performed on the radar point cloud images using pre-trained temporal convolutional neural networks and frequency-domain convolutional neural networks, respectively, to obtain the target recognition results. Since radar point cloud images contain one or more targets, the convolutional neural networks, after performing target detection on the radar point cloud images, obtain the recognition results of one or more targets. These recognition results are then listed to obtain a target list. Specifically, the target list obtained by the temporal convolutional neural network for target detection on the radar point cloud images is used as the first target list, and the target list obtained by the frequency-domain convolutional neural network for target detection on the radar point cloud images is used as the second target list.

[0063] Step S13: Match the first target list and the second target list to obtain the target recognition result of the radar point cloud image.

[0064] Both the first target list and the second target list contain the identification results of one or more targets. The identification results of the targets included in the two lists may be inconsistent, for example, the number of targets identified may be different (missed detection), or the classification information of the targets may be different. Therefore, it is necessary to match the first target list and the second target list. In fact, it is necessary to match the identification results of the targets included in the first target list and the second target list to obtain the matching identification results in the first target list and the second target list. The matching identification results are then used as the target identification results of the radar point cloud image for subsequent operations.

[0065] As can be seen from the above, the target recognition method provided in this embodiment of the invention first acquires a radar point cloud image collected by radar, and then performs target detection on the radar point cloud image based on pre-trained temporal convolutional neural networks and frequency-domain convolutional neural networks, respectively, to obtain a first target list and a second target list of the radar point cloud image. Both the first and second target lists include the recognition result of at least one target. The first and second target lists are then matched to obtain the target recognition result of the radar point cloud image. This embodiment of the invention performs target detection on the radar point cloud image based on both temporal and frequency-domain convolutional neural networks, and combines the recognition results from the temporal and frequency domains for comprehensive judgment. This can obtain more accurate target recognition results more efficiently, and better utilizes the features of different types of convolutional neural networks to assist in target recognition in the point cloud image, thereby improving the accuracy and efficiency of target recognition.

[0066] In one possible implementation, the above steps of matching the first target list and the second target list to obtain the target recognition result of the radar point cloud image include:

[0067] For any identification result in the first target list, if there is a matching identification result in the second target list, that identification result shall be taken as the target identification result.

[0068] In one possible implementation, the above step of matching the first target list and the second target list to obtain the target recognition result of the radar point cloud image further includes:

[0069] When there are mismatched identification results in the first target list and the second target list, time-frequency analysis is performed on the target to be processed corresponding to the mismatched identification result to determine the target identification result.

[0070] Both the first and second target lists include the identification results of one or more targets. For any identification result in either the first or second target list, that result is matched against the identification results in the other target list. The purpose of this matching is to verify the accuracy of the identification results included in the first and second target lists. Therefore, the matching process verifies whether there is a consistent identification result in the other target list. A consistent identification result in both the first and second target lists is considered a matched result. Verifying whether the target identification results are consistent in the first and second target lists can involve verifying whether the number of targets (whether any were missed), classification information, environment, physical location, scale information, etc., are consistent.

[0071] Only identification results that match in both the first and second target lists can be used as target identification results for the radar point cloud image and output for subsequent operations. If an identification result in either the first or second target list is inconsistent with a result in the other target list, the identification result is considered a mismatch (matching fails), and the target corresponding to the mismatched identification result is treated as a pending target for time-frequency analysis to further determine its identification result. Finally, the matching identification results and the results obtained from time-frequency analysis of the pending targets corresponding to the mismatched identification results are combined to form the target identification result of the radar point cloud image.

[0072] As can be seen from the above, the target recognition method provided by the embodiments of the present invention matches any recognition result in the first target list and the second target list with each recognition result in the other target list, and performs time-frequency analysis on the targets corresponding to the non-matching recognition results to further determine the recognition result of the target. Finally, the recognition results obtained by performing time-frequency analysis on the matching recognition results and the targets to be processed corresponding to the non-matching recognition results are used together as the target recognition result of the radar point cloud image. This avoids the errors that may exist when the time-domain convolutional neural network and the frequency-domain convolutional neural network perform target detection on the radar point cloud image, and further improves the accuracy of target recognition.

[0073] In one embodiment of the present invention, the identification result includes: classification information of the target corresponding to the identification result; such as Figure 2 As shown in the figure, this embodiment of the invention provides a flowchart of a second target recognition method. The above steps involve performing time-frequency analysis on the target corresponding to the recognition result to determine the target recognition result, including:

[0074] Step S21: Obtain a local image of the target to be processed within a preset range in the radar point cloud image;

[0075] Step S22: Obtain the time-frequency spectrum of the local image and analyze the time-frequency spectrum to obtain the environmental information of the target corresponding to the recognition result.

[0076] The classification information of the targets mentioned above refers to the type of target, such as animals, plants, pedestrians, vehicles, signs, etc. The target identification result includes the target classification information, and may also include the target name. Specifically, for multiple targets with the same classification information, they can be distinguished by different target names. For example, when the classification information of two targets is pedestrian, the target names of these two targets can be distinguished as pedestrian A and pedestrian B.

[0077] The aforementioned local image is a local image within a preset range around the target to be processed in the radar point cloud image. The preset range is a pre-defined range that indicates that the local image within this range can display the environment where the target to be processed is located, and is determined according to actual needs.

[0078] The aforementioned time-frequency spectrum diagram of a local image refers to the time-frequency spectrum diagram of a local image of the target in a radar point cloud image, which may include phase spectrum and amplitude spectrum. Based on the calculated and compared spectral content in the time-frequency spectrum diagram, environmental information about the surrounding environment of the target can be obtained. For example, environmental information includes roads, sky, etc.

[0079] Step S23: Determine the attribute information of the target to be processed based on the time-frequency spectrum diagram.

[0080] The attribute information includes geometric shape, and the time-frequency spectrum includes instantaneous frequency and local group delay information.

[0081] Step S24: Determine the classification information of the target to be processed based on the attribute information.

[0082] In one embodiment of the present invention, at least one type of information, including sine waves, linear chirps, superimposed linear chirps, and specific frequency pulses, can be derived from the time-frequency spectrum. These are specific attributes that are difficult to determine by convolutional neural networks. Therefore, target detection based on the aforementioned convolutional neural network may contain errors. These errors can be determined by calculating the instantaneous frequency and local group delay of the aforementioned time-frequency spectrum. The attribute information includes at least the geometric shape of the target to be processed, and may also include other attributes related to the target characteristics. Based on the attribute information, the classification information of the target to be processed can be further determined. For example, if the convolutional neural network identifies the classification information of the target to be processed as an animal, the attribute information can be used to determine whether the target is specifically a cat, dog, etc.

[0083] Step S25: If the environmental information and the classification information meet the preset matching principle, the classification information is used as the target recognition result.

[0084] After determining the classification and environmental information of the target to be processed, the two need to be matched. Only if the environmental information and classification information meet the preset matching principles will the classification information be used as the target identification result. For example, the environmental information of land animals (cats, dogs, etc.) should not be the sky, and the environmental information of pedestrians should not be trees, buildings, etc. If the classification and environmental information of the target to be processed do not meet the preset matching principles, the identification result is considered incorrect. Only if the classification and environmental information of the target to be processed meet the preset matching principles will the identification result be considered accurate and used as the target identification result, that is, the identification result of the target to be processed.

[0085] The aforementioned preset matching principles are various types of environmental information corresponding to different categories of targets, which can be set according to actual needs.

[0086] In one example, in intelligent transportation applications, traffic accidents such as car crashes may occur when targets in radar point cloud images cannot be detected and identified. For instance, during the detection and identification process, a temporal convolutional network might identify the license plate of a vehicle in front of it as a bright spot and assume that the sun is the only bright object that matches this spot. Consequently, it fails to identify the license plate to determine if a vehicle is ahead, leading to an incorrect decision and ultimately a traffic accident. When a time-domain convolutional neural network fails to recognize a license plate, but a frequency-domain convolutional network detects and identifies the license plate (a bright spot), it performs time-frequency analysis on the time-frequency spectrum of the local image containing the bright spot (specifically, an image with the center point of the bright spot as the origin and a preset distance as the radius, such as 5m). This analysis determines the environmental information of the bright spot, identifying its boundaries and thus confirming that the bright spot is located between the two boundaries of the vehicle, with the car tires below it. Furthermore, by calculating the instantaneous frequency and local group delay of the time-frequency spectrum of the local image containing the bright spot, it further obtains the attribute information of the bright spot. For example, by comparing the superimposed linear chirp values ​​captured from the surrounding local images, it makes a more accurate estimate of the bright spot's geometric features. Combining these two pieces of information (first, determining that the bright spot is located behind the vehicle, between its boundaries; second, confirming that the geometric dimensions of the bright spot match the size range of a license plate), it is considered reasonable for the license plate to appear at the current location, meaning it conforms to the preset matching rules, and the confidence level that the bright spot is a license plate is high.

[0087] In one embodiment of the present invention, the environmental information includes environmental objects;

[0088] The step of using the classification information as the target recognition result when the environmental information and the classification information meet a preset matching principle further includes:

[0089] Determine whether the relative distance between the target to be processed and the environmental object meets a preset distance standard. If so, determine that the environmental information and the classification information conform to a preset matching principle. The environmental object includes road signs and / or at least one target in the first target list and / or the second target list other than the target to be processed.

[0090] Since the traffic environment is the result of the interaction of a series of factors such as people, vehicles, roads, and buildings on the road, the relative positional relationship between the target and nearby targets can be further considered on the basis of the above matching rules to further improve the accuracy of recognition. That is, environmental information includes environmental objects, which include road signs and / or at least one target in the first target list and / or the second target list other than the target to be processed. In other words, the environmental information here is the environment in which the target to be processed is located. Broadly speaking, it includes all factors within a certain range of the target to be processed, including both static background environmental information and other targets in the same scene as the target to be processed.

[0091] Then, the step of using the classification information as the target recognition result when the environmental information and the classification information meet the preset matching principle further includes:

[0092] Determine whether the relative distance between the target to be processed and nearby targets meets the preset distance standard. If so, determine that the environmental information and the classification information of the target to be processed conform to the preset matching principle.

[0093] In this embodiment, road signs may include road facilities such as signs, mailboxes, green belts, buildings, etc. Other targets in the first target list and / or the second target list, besides the target to be processed, may include people, adjacent vehicles, etc. Considering the height of the vehicle with the license plate, the need for safe driving and roadside clearance, and the distance that vehicles in adjacent lanes need to maintain, which are limited by traffic rules and the actual physical environment, there is a certain preset distance standard between the license plate and the aforementioned environmental objects. Therefore, the distance between the license plate and the environmental objects can be calculated to determine whether it exceeds or is less than the preset distance standard. If it is within the normal range, the assessment result is considered reliable.

[0094] In this embodiment, different preset distance standards can be set according to different environmental objects. For example, the preset distance standards between the license plate and the ground, between the license plate and roadside facilities (such as mailboxes or green belts), and between the license plate and other nearby vehicles are different; the preset distance standards between the license plate of a car and the license plate of a large truck and the ground are also not completely the same.

[0095] Therefore, based on the time-frequency analysis method, which comprehensively considers the geometric characteristics of the target to be processed, its environment, and its relative positional relationship with other targets in the environment, a more reliable target recognition result can be given, eliminating the error caused by applying a single time-domain or frequency-domain convolutional network.

[0096] In one example, the target recognition method described above can be applied to a radar system mounted on an intelligent vehicle (autonomous vehicle). Specifically, it can be applied to an ASIC (Application Specific Integrated Circuit) readout chip or FPGA (Field Programmable Gate Array) included in the radar system. The ASIC or FPGA can include a storage module, a computing module, and an input / output module to implement the target recognition method. The computing module is used to pre-train a temporal convolutional neural network and a frequency-domain convolutional neural network using a publicly available radar point cloud image database. It is also used to perform relevant calculations for target detection and recognition on the radar point cloud image using the temporal and frequency-domain convolutional neural networks. The storage module stores the coefficients of the pre-trained temporal and frequency-domain convolutional neural networks. The input / output module interacts with the radar system, receiving radar point cloud images captured by a SPAD (Single Photon Avalanche Diode) in the radar system, and receiving and outputting the target recognition results from the radar point cloud images.

[0097] Specifically, the computation module may also include sub-modules for training temporal convolutional neural networks and frequency-domain convolutional neural networks. Specifically, these include: a sub-module for calculating the two-dimensional fast Fourier transform of radar point cloud images in a publicly available radar point cloud image database; a sub-module for calculating the coefficients of the frequency-domain convolutional neural network based on the two-dimensional Fourier transform results of three-dimensional radar point cloud images; and a sub-module for calculating the coefficients of the temporal convolutional neural network based on radar point cloud images in a publicly available radar point cloud image database.

[0098] In addition, the calculation module may also include a sub-module for target detection and recognition of radar point cloud images based on temporal convolutional neural networks and frequency convolutional neural networks. Specifically, it includes: a sub-module for target detection of radar point cloud images based on temporal convolutional neural networks and frequency convolutional neural networks; a sub-module for post-processing the first target list and the second target list generated by the temporal convolutional neural network and the frequency convolutional neural network; and a sub-module for obtaining the target recognition result of the radar point cloud image based on the first target list and the second target list.

[0099] The storage module may also include a sub-module for updating and storing the coefficients of the time-domain convolutional neural network and the frequency-domain convolutional neural network, and may also include a FIFO (First Input First Output) module, which can act as a buffer to store the first target list and the second target list.

[0100] In one example, the aforementioned temporal and frequency-domain convolutional neural networks have a traditional convolutional neural network structure, including an input layer, seven hidden layers, and an output layer. The input layer consists of a column of vectors p, representing scalar values ​​calculated from radar point cloud images in a publicly available radar point cloud image database. The hidden layers all include weights (LW), multiplication and summation operations, a radix b, and a transfer function f, used to calculate the equation: a = f(LW*p + b). The output layer is the layer that produces the output of the convolutional neural network. Specifically, the analytical expression for the output of each of the seven hidden layers can be as follows:

[0101] a 1 =f 1 (LW 1,1 *p+b 1 )

[0102] a 2 =f 2 (LW 2,1 *a 1 +b 2 )

[0103] a 3 =f 3 (LW 3,2 *a 2 +b 3 )

[0104] a 4 =f 4 (LW 4,2 *a 3 +b 4 )

[0105] a 5 =f 5 (LW 5,2 *a 4 +b 5 )

[0106] a 6 =f 6 (LW 6,2 *a 5 +b 6 )

[0107] a 7 =f 7 (LW 7,2 *a 6 +b 7 )

[0108] Among them, LW x,y The weights are represented by x, x represents the x-th layer, y represents the number of nodes, and a represents the weights. xThis represents the output of layer x. As can be seen from the above, the target recognition method provided in this embodiment of the invention, after obtaining the matching situation of the first target list and the second target list, performs time-frequency analysis on the target to be processed corresponding to the mismatched recognition results to further determine the recognition result of the target. At the same time, it considers the environmental information and attribute information of the target to be processed, as well as the degree of matching between the two, and avoids the possible errors in the target detection and recognition process from the aspects of environment and attributes, thereby further improving the accuracy of target recognition.

[0109] See Figure 3 The present invention also provides a schematic diagram of a target recognition device, the device comprising:

[0110] Image acquisition module 301 is used to acquire radar point cloud images collected by radar;

[0111] The target list acquisition module 302 is used to perform target detection on the radar point cloud image based on a pre-trained time-domain convolutional neural network and a frequency-domain convolutional neural network, respectively, to obtain a first target list and a second target list of the radar point cloud image, wherein the first target list and the second target list each include the recognition result of at least one target.

[0112] The identification result determination module 303 is used to match the first target list and the second target list to obtain the target identification result of the radar point cloud image.

[0113] As can be seen from the above, the target recognition device provided in this embodiment of the invention first acquires a radar point cloud image collected by radar, and then performs target detection on the radar point cloud image based on pre-trained temporal convolutional neural networks and frequency-domain convolutional neural networks, respectively, to obtain a first target list and a second target list of the radar point cloud image. Both the first and second target lists include the recognition result of at least one target. The first and second target lists are then matched to obtain the target recognition result of the radar point cloud image. This embodiment of the invention performs target detection on the radar point cloud image based on both temporal and frequency-domain convolutional neural networks, and combines the recognition results from the temporal and frequency domains for comprehensive judgment. This can obtain more accurate target recognition results more efficiently, and better utilizes the features of different types of convolutional neural networks to assist in target recognition in the point cloud image, thereby improving the accuracy and efficiency of target recognition.

[0114] In one embodiment of the present invention, the identification result determination module 303 is specifically used for:

[0115] For any identification result in the first target list, if there is a matching identification result in the second target list, that identification result shall be taken as the target identification result.

[0116] In one embodiment of the present invention, the identification result determination module 303 is specifically used for:

[0117] When there are mismatched identification results between the first target list and the second target list, time-frequency analysis is performed on the target to be processed corresponding to the mismatched identification result to determine the target identification result.

[0118] As can be seen from the above, the target recognition device provided in this embodiment of the invention matches any recognition result in the first target list and the second target list with each recognition result in the other target list, and performs time-frequency analysis on the targets corresponding to the non-matching recognition results to further determine the recognition result of the target. Finally, the recognition results obtained by performing time-frequency analysis on the matching recognition results and the targets to be processed corresponding to the non-matching recognition results are used together as the target recognition result of the radar point cloud image. This avoids the errors that may exist when the time-domain convolutional neural network and the frequency-domain convolutional neural network perform target detection on the radar point cloud image, and further improves the accuracy of target recognition.

[0119] In one embodiment of the present invention, the identification result includes: classification information of the target corresponding to the identification result; the identification result determination module 303 is specifically used for:

[0120] Acquire a local image of the target to be processed within a preset range in the radar point cloud image;

[0121] The time-frequency spectrum of the local image is obtained, and the environmental information of the target to be processed is obtained by analyzing the time-frequency spectrum.

[0122] The attribute information of the target to be processed is determined based on the time-frequency spectrum, wherein the attribute information includes geometric shape;

[0123] The classification information of the target to be processed is determined based on the attribute information;

[0124] If the environmental information and the classification information meet the preset matching principle, the classification information is used as the target identification result.

[0125] In one embodiment of the present invention, the environmental information includes environmental objects; the identification result determination module 303 is specifically used for:

[0126] Determine whether the relative distance between the target to be processed and the environmental object meets a preset distance standard. If so, determine that the environmental information and the classification information conform to a preset matching principle. The environmental object includes road signs and / or at least one target in the first target list and / or the second target list other than the target to be processed.

[0127] In one embodiment of the present invention, at least one type of information, including sine wave, linear chirp, superimposed linear chirp, and pulse of a specific frequency, can be obtained from the time-frequency spectrum diagram.

[0128] As can be seen from the above, the target recognition device provided in this embodiment of the invention, after obtaining the matching status of the first target list and the second target list, performs time-frequency analysis on the target to be processed corresponding to the mismatched recognition results to further determine the recognition result of the target. At the same time, it considers the environmental information and attribute information of the target to be processed as well as the degree of matching between the two, thus avoiding possible errors in the target detection and recognition process from the aspects of environment and attributes, and further improving the accuracy of target recognition.

[0129] This invention also provides an electronic device, such as... Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0130] Memory 403 is used to store computer programs;

[0131] When the processor 401 executes the program stored in the memory 403, it implements any of the above-described target recognition method steps.

[0132] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0133] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0134] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0135] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0136] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the target recognition methods described above.

[0137] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the target recognition methods described above.

[0138] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0139] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0140] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0141] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A target recognition method, characterized in that, The method includes: Acquire radar point cloud images collected by radar; Based on pre-trained temporal convolutional neural networks and frequency convolutional neural networks, target detection is performed on the radar point cloud image to obtain a first target list and a second target list of the radar point cloud image, wherein the first target list and the second target list each include the recognition result of at least one target. The first target list and the second target list are matched to obtain the target recognition result of the radar point cloud image; The step of matching the first target list and the second target list to obtain the target recognition result of the radar point cloud image includes: When there is a mismatch between the identification results of the first target list and the second target list, a local image of the target to be processed within a preset range in the radar point cloud image is obtained; The time-frequency spectrum of the local image is obtained, and the environmental information of the target to be processed is obtained by analyzing the time-frequency spectrum. The attribute information of the target to be processed is determined based on the time-frequency spectrum, wherein the attribute information includes geometric shape; The classification information of the target to be processed is determined based on the attribute information; If the environmental information and the classification information meet the preset matching principle, the classification information is used as the target identification result.

2. The method according to claim 1, characterized in that, The step of matching the first target list and the second target list to obtain the target recognition result of the radar point cloud image includes: For any identification result in the first target list, if there is a matching identification result in the second target list, that identification result shall be taken as the target identification result.

3. The method according to claim 1, characterized in that, The environmental information includes environmental objects; The step of using the classification information as the target recognition result when the environmental information and the classification information meet a preset matching principle further includes: Determine whether the relative distance between the target to be processed and the environmental object meets a preset distance standard. If so, determine that the environmental information and the classification information conform to a preset matching principle. The environmental object includes road signs and / or at least one target in the first target list and / or the second target list other than the target to be processed.

4. A target recognition device, characterized in that, The device includes: The image acquisition module is used to acquire radar point cloud images collected by the radar. The target list acquisition module is used to perform target detection on the radar point cloud image based on a pre-trained temporal convolutional neural network and a frequency convolutional neural network, respectively, to obtain a first target list and a second target list of the radar point cloud image, wherein the first target list and the second target list each include the recognition result of at least one target; The identification result determination module is used to match the first target list and the second target list to obtain the target identification result of the radar point cloud image; The identification result determination module is specifically used for: When there is a mismatch between the identification results of the first target list and the second target list, a local image of the target to be processed within a preset range in the radar point cloud image is obtained; The time-frequency spectrum of the local image is obtained, and the environmental information of the target to be processed is obtained by analyzing the time-frequency spectrum. The attribute information of the target to be processed is determined based on the time-frequency spectrum, wherein the attribute information includes geometric shape; The classification information of the target to be processed is determined based on the attribute information; If the environmental information and the classification information meet the preset matching principle, the classification information is used as the target identification result.

5. The apparatus according to claim 4, characterized in that, The identification result determination module is specifically used for: For any identification result in the first target list, if there is a matching identification result in the second target list, that identification result shall be taken as the target identification result.

6. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-3.

Images