Target perception identification method, system and device, and medium
By using weather features in target detection for weight adaptive calculation and weighted fusion feature processing, the problem of reduced accuracy of target detection in bad weather is solved, and the accuracy and reliability of target sense knowledge are improved.
Patent Information
- Application Number
- CN202411376895.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-01-03
AI Technical Summary
The existing target detection methods have high noise in sensor data under severe weather conditions, resulting in a decrease in the accuracy of target sense knowledge and failing to effectively adapt to different weather conditions.
By obtaining multiple sensor data, target features are extracted and weighted adaptive calculations are used using weather features, each target feature has different weights, and weighted fusion feature processing is performed, and ultimately, target sense knowledge is distinguished based on weighted fusion features.
It improves the accuracy and reliability of target knowledge and can better adapt to different weather conditions.
Smart Images
Figure CN119249354A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vehicle-road collaboration, and in particular to a target perception and recognition method, system, device and medium. Background Art
[0002] In order to better obtain information about road traffic participants, multiple different types of sensors are usually installed and deployed on the roadside, such as lidar, millimeter-wave radar, cameras, etc. Target detection is performed through the data collected by the sensors, so that effective coordination between people, vehicles and roads can be achieved based on the target detection results.
[0003] The current target detection method is to simply fuse data from multiple sensors and then use the perception model to identify the fused features. However, the above method does not take into account the limitations of sensors under different weather conditions. For example, in bad weather such as rain, fog, and snow, the performance of lidar may be affected and the data it collects will have high noise. The performance of video cameras will drop significantly under poor lighting conditions.
[0004] Therefore, under different weather conditions, simply fusing data from multiple sensors cannot guarantee the accuracy of target perception and recognition. Summary of the invention
[0005] In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present invention proposes a target perception and recognition method, comprising the following steps: Acquire multiple sensor data of a target and extract multiple target features from the multiple sensor data; Obtaining weather characteristics using first sensor data from the plurality of sensor data and performing weight adaptive calculation based on the weather characteristics to obtain a weight corresponding to each target characteristic; A weighted fusion feature is obtained based on the multiple target features and the corresponding weights, and target perception and recognition is performed based on the weighted fusion feature.
[0006] In some embodiments, acquiring multiple sensor data of a target and extracting multiple target features from the multiple sensor data further includes: Using a variety of sensors to collect data on the target to obtain the multiple sensor data; A plurality of encoders are used to respectively extract features from a type of sensor data to obtain the plurality of target features.
[0007] In some embodiments, obtaining weather features using first sensor data from the plurality of sensor data and performing weight adaptive calculation based on the weather features to obtain weights corresponding to each target feature further comprises: extracting features from the first sensor data using a first encoder to obtain the weather features and calculating weights corresponding to each target feature based on the weather features using a weight adaptive extraction network; A weighted fusion feature is obtained based on the multiple target features and the corresponding weights, and target perception and recognition is performed based on the weighted fusion feature, further including: multiplying each target feature by the corresponding weight and then fusing them to obtain the weighted fusion feature, and using a perception model to perform target perception and recognition based on the weighted fusion feature.
[0008] In some embodiments, it also includes: The plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model are trained.
[0009] In some embodiments, training the plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model further comprises: Constructing a first training set and a second training set, wherein the first training set includes the multiple sensor data, and the second training set includes the first sensor data; Using the first training set to train the plurality of encoders and the perception model, and using the second training set to train the first encoder and the weight adaptive extraction network; Based on the first loss function and the second loss function, the multiple encoders, the first encoder, the weight adaptive extraction network, the perception model, and the weather recognition network are iteratively optimized until the first loss function and the second loss function meet preset requirements, and the training is stopped.
[0010] In some embodiments, training the first encoder and the weight adaptive extraction network further includes: constructing a third training set, wherein the third training set includes the first sensor data; Using the third training set to train the first encoder and the weight adaptive extraction network; The first encoder, the weight adaptive extraction network and the weather recognition network are iteratively optimized based on the third loss function until the third loss function reaches a preset requirement and the training is stopped.
[0011] In some embodiments, training the plurality of encoders and the perceptual model further comprises: constructing a fourth training set, wherein the fourth training set includes the multiple sensor data; Using the fourth training set to train the plurality of encoders and the perception model; The plurality of encoders and the perception model are iteratively optimized based on the fourth loss function until the fourth loss function meets a preset requirement and the training is stopped.
[0012] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a target perception and recognition system, including: a feature extraction module configured to obtain a plurality of sensor data of a target and extract a plurality of target features from the plurality of sensor data; A calculation module, configured to obtain weather characteristics using first sensor data among the plurality of sensor data and to perform weight adaptive calculation based on the weather characteristics to obtain a weight corresponding to each target characteristic; The recognition module is configured to obtain a weighted fusion feature based on the multiple target features and the corresponding weights and perform target perception recognition based on the weighted fusion feature.
[0013] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer device, including: at least one processor; and A memory, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps of any one of the target perception and recognition methods described above are performed.
[0014] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the target perception and recognition methods described above are performed.
[0015] The present invention has one of the following beneficial technical effects: the scheme proposed in the present invention utilizes weather characteristics to obtain weight values, and assigns different weights to different target characteristics, so as to calculate weighted fusion features based on the weight values, and finally perform perception and recognition based on the weighted fusion features, which can better adapt to different weather conditions, thereby improving the accuracy and reliability of target perception and recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.
[0017] Figure 1 A schematic diagram of a flow chart of a target perception and recognition method provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a target perception and recognition system provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION
[0018] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0019] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.
[0020] According to one aspect of the present invention, an embodiment of the present invention provides a target perception and recognition method, such as Figure 1 As shown, it may include the steps of: S1, acquiring multiple sensor data of a target and extracting multiple target features from the multiple sensor data; S2, obtaining weather characteristics using first sensor data among the plurality of sensor data and performing weight adaptive calculation based on the weather characteristics to obtain a weight corresponding to each target characteristic; S3, obtaining a weighted fusion feature based on the multiple target features and the corresponding weights, and performing target perception and recognition based on the weighted fusion feature.
[0021] The solution proposed in the present invention utilizes weather characteristics to obtain weight values, and assigns different weights to different target features, so as to calculate weighted fusion features based on the weight values, and finally perform perception and recognition based on the weighted fusion features. This can better adapt to different weather conditions, thereby improving the accuracy and reliability of target perception and recognition.
[0022] In some embodiments, acquiring multiple sensor data of a target and extracting multiple target features from the multiple sensor data further includes: Using a variety of sensors to collect data on the target to obtain the multiple sensor data; A plurality of encoders are used to respectively extract features from a type of sensor data to obtain the plurality of target features.
[0023] Specifically, the sensor can be an image sensor, a laser radar, a millimeter-wave radar, etc. Different types of sensors have their own unique advantages. For example, a laser radar can provide very high spatial resolution and has high accuracy in detecting the shape, size, and distance of an object. Image sensors can provide rich color information, which helps to identify objects and scenes. Millimeter-wave radars are particularly sensitive to metal objects, so using multiple sensors to collect targets is conducive to extracting a more comprehensive feature representation of the target.
[0024] After using multiple sensors to collect data on a target, the original data can be first converted into a matrix, and then the matrix can be converted into a vector through an explicit projection, and then the corresponding vectors can be processed in parallel by multiple encoders, so that rich target features can be extracted from different sensor data at the same time. For example, the image data collected by the image sensor is subjected to feature extraction by the second encoder to obtain image features (first target features), such as image BEV features, the laser radar point cloud data collected by the laser radar is subjected to feature extraction by the third encoder to obtain laser radar point cloud features (second target features), such as laser radar point cloud BEV features, and the millimeter wave radar point cloud data collected by the millimeter wave radar is subjected to feature extraction by the fourth encoder to obtain millimeter wave radar point cloud features (third target features), such as millimeter wave radar point cloud BEV features. In some embodiments, the second encoder, the third encoder, and the fourth encoder can be convolutional neural networks or Transformers.
[0025] In some embodiments, obtaining weather characteristics using first sensor data among the plurality of sensor data and performing weight adaptive calculation based on the weather characteristics to obtain weights corresponding to each target characteristic further includes: The first encoder is used to perform feature extraction on the first sensor data to obtain the weather features, and a weight adaptive extraction network is used to calculate the weight corresponding to each target feature based on the weather features.
[0026] Specifically, the first encoder can be a machine learning model, such as a convolutional neural network or a Transformer, which extracts information that can represent weather characteristics from the first sensor data through the first encoder. For example, if the first sensor data is image data, the first encoder is used to extract weather characteristics from areas such as the sky in the image data. In some embodiments, the weight adaptive extraction network can be an MLP (multi-layer perceptron), which uses the MLP (multi-layer perceptron) to calculate the weather characteristics to obtain the weight corresponding to each target feature.
[0027] The weight adaptive extraction network is a model used to calculate the weight corresponding to each target feature according to weather characteristics. It takes weather characteristics as input and outputs the weight of each target feature through a series of calculations and mappings. This process is adaptive, so that the weights are adjusted according to different weather conditions. That is, in bad weather, some target features may become more important, so their weights will increase accordingly; in good weather, other features may be more representative, and the weights will be adjusted accordingly. For example, if the above-mentioned image sensors, lidars, and millimeter-wave radars are used for data collection, when the weather is good, the lidars and image sensors can meet most of the needs, and the weight of the third target feature (obtained based on millimeter-wave radar point cloud data) can be reduced; when the weather conditions are bad, the weight of the third target feature can be increased, and the weight of the first target feature (obtained based on image data) and the weight of the second target feature (obtained based on lidar) can be reduced.
[0028] In some embodiments, obtaining a weighted fusion feature based on the multiple target features and the corresponding weights and performing target perception recognition based on the weighted fusion feature further includes: Each of the target features is multiplied by the corresponding weight and then fused to obtain the weighted fusion feature, and the perception model is used to perform target perception and recognition based on the weighted fusion feature.
[0029] Specifically, since the weight reflects the importance of the corresponding target feature under specific weather conditions, for each target feature, it is multiplied by the corresponding weight so that the target feature is adjusted according to its weight. Important features will get larger values after weighting, while relatively unimportant features will be weakened. Finally, all weighted target features are fused. There are many ways to fusion, such as simple linear weighted fusion, that is, multiplying all target features by weights to obtain fused features; or using more complex fusion strategies, such as neural network-based fusion layers, which automatically learn the best fusion method between different features.
[0030] In some embodiments, the perception model may include a 3D convolution module, a decoder module and a predictive feedforward neural network module. The 3D target detection result can be obtained by calculating the weighted fusion features through the above modules in sequence.
[0031] In some embodiments, it also includes: The plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model are trained.
[0032] Specifically, the multiple encoders, the first encoder, the weight adaptive extraction network, and the perception model can be trained jointly, or the first encoder and the weight adaptive extraction network can be trained separately, and the multiple encoders and the perception model can be trained separately.
[0033] In some embodiments, training the plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model further comprises: Constructing a first training set and a second training set, wherein the first training set includes the multiple sensor data, and the second training set includes the first sensor data; Using the first training set to train the plurality of encoders and the perception model, and using the second training set to train the first encoder and the weight adaptive extraction network; Based on the first loss function and the second loss function, the multiple encoders, the first encoder, the weight adaptive extraction network, the perception model, and the weather recognition network are iteratively optimized until the first loss function and the second loss function meet preset requirements, and the training is stopped.
[0034] Specifically, when the plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model are jointly trained, a first training set including multiple sensor data and a second training set including the first sensor data can be constructed. For example, the first training set can include image data, lidar point cloud data, and millimeter wave radar point cloud data, while the second training set includes image data. Since the roadside sensors are fixed, the roadside images can be manually or automatically selected for fixed areas of interest. For example, in the second training set, areas that are more related to the weather, such as the sky, can be manually found, and areas such as buildings that are irrelevant or not very relevant to the weather can be removed. Semantic segmentation models, such as Mask2Former, can also be used to automatically remove areas such as buildings that are irrelevant or not very relevant to the weather.
[0035] Then, the first training set and the second training set are used for joint training. During the training, the various sensor data in the first training set are respectively input into the corresponding encoders for feature extraction to obtain multiple target features; the first sensor data in the second training set is input into the first encoder for feature extraction to obtain weather features; the weather features are input into the weight adaptive extraction network to obtain the weights corresponding to each target feature; each target feature is multiplied by the corresponding weight and then fused to obtain weighted fusion features. Through the above process, each weight and weighted fusion feature can be obtained. Finally, each weight calculation is input into the weather recognition network to calculate the second loss function according to the output of the weather recognition network, and the weighted fusion feature is input into the perception model and the first loss function is calculated based on the output of the perception model.
[0036] Finally, the first loss function and the second loss function are assigned different weights and added as the overall loss function. Based on the overall loss function, the gradient backpropagation algorithm is used to jointly iteratively optimize multiple encoders, the first encoder, the weight adaptive extraction network, the perception model and the weather recognition network. By continuously adjusting the parameters of these models, the overall loss function is gradually reduced, and the performance of each model is improved. The preset requirements may be that the overall loss function is less than a certain threshold, the overall loss function no longer decreases significantly, or reaches a certain number of iterations. When the preset requirements are met, it means that the model has been trained well enough and can be used for actual target perception and recognition tasks. In some embodiments, the first loss function and the second loss function may include multiple loss functions such as the cross entropy loss function.
[0037] In some embodiments, the weather recognition network may be a weather recognition classifier, which may be a single-layer fully connected layer.
[0038] In some embodiments, training the first encoder and the weight adaptive extraction network further includes: constructing a third training set, wherein the third training set includes the first sensor data; Using the third training set to train the first encoder and the weight adaptive extraction network; The first encoder, the weight adaptive extraction network and the weather recognition network are iteratively optimized based on the third loss function until the third loss function reaches a preset requirement and the training is stopped.
[0039] Specifically, when the first encoder and the weight adaptive extraction network are trained separately, a third training set including the first sensor data can be constructed. For example, the third training set includes image data. Moreover, since the roadside sensors are fixed, the roadside images can manually or automatically select fixed areas of interest. For example, in the second training set, areas that are more related to the weather, such as the sky, can be manually found, and areas such as buildings that are irrelevant or not very relevant to the weather can be removed. Semantic segmentation models, such as Mask2Former, can also be used to automatically remove areas such as buildings that are irrelevant or not very relevant to the weather.
[0040] Then, the third training set is used for training. During training, the first sensor data in the third training set is input into the first encoder for feature extraction to obtain weather features; the weather features are input into the weight adaptive extraction network to obtain the weight corresponding to each target feature; each weight calculation is input into the weather recognition network to calculate the first loss function according to the output of the weather recognition network Finally, based on the first loss function, the gradient backpropagation algorithm is used to iteratively optimize the first encoder, the weight adaptive extraction network and the weather recognition network. By continuously adjusting the parameters of these models, the first loss function is gradually reduced, and the performance of each model is improved. The preset requirements may be that the first loss function is less than a certain threshold, the first loss function no longer decreases significantly, or a certain number of iterations is reached. When the preset requirements are met, it means that the model has been trained well enough and can be used for actual reasoning tasks. It should be noted that the weather recognition network does not need to participate in the actual reasoning and calculation of weights. In some embodiments, the first loss function may include multiple loss functions such as the cross entropy loss function.
[0041] In some embodiments, training the plurality of encoders and the perceptual model further comprises: constructing a fourth training set, wherein the fourth training set includes the at least one type of sensor data; Using the fourth training set to train the plurality of encoders and the perception model; The plurality of encoders and the perception model are iteratively optimized based on the fourth loss function until the fourth loss function meets a preset requirement and the training is stopped.
[0042] Specifically, when multiple encoders and the perception models are jointly trained separately, a fourth training set including multiple sensor data can be constructed. For example, the fourth training set may include image data, lidar point cloud data, and millimeter-wave radar point cloud data.
[0043] Then, the fourth training set is used for training. During the training, the various sensor data in the fourth training set are respectively input into the corresponding encoders for feature extraction to obtain multiple target features; the first sensor data in the fourth training set is input into the first encoder for feature extraction to obtain weather features; the weather features are input into the weight adaptive extraction network to obtain the weights corresponding to each target feature; each target feature is multiplied by the corresponding weight and then fused to obtain weighted fusion features. Finally, the weighted fusion features are input into the perception model and the fourth loss function is calculated based on the output of the perception model.
[0044] Finally, based on the fourth loss function, the gradient backpropagation algorithm is used to iteratively optimize multiple encoders and perception models. By continuously adjusting the parameters of these models, the fourth loss function is gradually reduced, and the performance of each model is improved. The preset requirements may be that the fourth loss function is less than a certain threshold, the fourth loss function no longer decreases significantly, or reaches a certain number of iterations. When the preset requirements are met, it means that the model has been trained well enough and can be used for actual target perception and recognition tasks. In some embodiments, the fourth loss function may include multiple loss functions such as a cross entropy loss function.
[0045] The solution proposed in the present invention utilizes weather characteristics to obtain weight values, and assigns different weights to different target features, so as to calculate weighted fusion features based on the weight values, and finally perform perception and recognition based on the weighted fusion features. This can better adapt to different weather conditions, thereby improving the accuracy and reliability of target perception and recognition.
[0046] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a target perception and recognition system 400, such as Figure 2 As shown, including: A feature extraction module 401 is configured to obtain multiple sensor data of a target and extract multiple target features from the multiple sensor data; A calculation module 402 is configured to obtain weather characteristics using first sensor data among the plurality of sensor data and to perform weight adaptive calculation based on the weather characteristics to obtain a weight corresponding to each target characteristic; The recognition module 403 is configured to obtain a weighted fusion feature based on the multiple target features and the corresponding weights and perform target perception recognition based on the weighted fusion feature.
[0047] In some embodiments, the feature extraction module 401 is further configured to: Using a variety of sensors to collect data on the target to obtain the multiple sensor data; A plurality of encoders are used to respectively extract features from a type of sensor data to obtain the plurality of target features.
[0048] In some embodiments, the calculation module 402 is further configured to use a first encoder to perform feature extraction on the first sensor data to obtain the weather feature and use a weight adaptive extraction network to calculate a weight corresponding to each target feature based on the weather feature; The recognition module 403 is further configured to: multiply each of the target features by the corresponding weights and fuse them to obtain the weighted fusion features, and use the perception model to perform target perception recognition based on the weighted fusion features.
[0049] In some embodiments, a training module is further included, configured to: The plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model are trained.
[0050] In some embodiments, the training module is further configured to: Constructing a first training set and a second training set, wherein the first training set includes the multiple sensor data, and the second training set includes the first sensor data; Using the first training set to train the plurality of encoders and the perception model, and using the second training set to train the first encoder and the weight adaptive extraction network; Based on the first loss function and the second loss function, the multiple encoders, the first encoder, the weight adaptive extraction network, the perception model, and the weather recognition network are iteratively optimized until the first loss function and the second loss function meet preset requirements, and the training is stopped.
[0051] In some embodiments, the training module is further configured to: constructing a third training set, wherein the third training set includes the first sensor data; Using the third training set to train the first encoder and the weight adaptive extraction network; The first encoder, the weight adaptive extraction network and the weather recognition network are iteratively optimized based on the third loss function until the third loss function reaches a preset requirement and the training is stopped.
[0052] In some embodiments, the training module is further configured to: constructing a fourth training set, wherein the fourth training set includes the at least one type of sensor data; Using the fourth training set to train the plurality of encoders and the perception model; The plurality of encoders and the perception model are iteratively optimized based on the fourth loss function until the fourth loss function meets a preset requirement and the training is stopped.
[0053] Based on the same inventive concept, according to another aspect of the present invention, Figure 3 As shown, an embodiment of the present invention further provides a computer device 501, including: at least one processor 520; and The memory 510 stores a computer program 511 that can be run on the processor. When the processor 520 executes the program, the steps of any one of the above target perception and recognition methods are performed.
[0054] Based on the same inventive concept, according to another aspect of the present invention, Figure 4 As shown, an embodiment of the present invention further provides a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, the steps of any of the above target perception and recognition methods are performed.
[0055] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.
[0056] Furthermore, it should be appreciated that the computer-readable storage medium (eg, memory) herein may be either volatile memory or nonvolatile memory, or may include both volatile memory and nonvolatile memory.
[0057] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.
[0058] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.
[0059] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.
[0060] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0061] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0062] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.
Claims
1. A target perception and recognition method, characterized in that: The following steps are involved: Acquire multiple sensor data of a target and extract multiple target features from the multiple sensor data; Obtaining weather characteristics using first sensor data from the plurality of sensor data and performing weight adaptive calculation based on the weather characteristics to obtain a weight corresponding to each target characteristic; A weighted fusion feature is obtained based on the multiple target features and the corresponding weights, and target perception and recognition is performed based on the weighted fusion feature.
2. The method according to claim 1, characterized in that Acquiring multiple sensor data of a target and extracting multiple target features from the multiple sensor data further includes: Using a variety of sensors to collect data on the target to obtain the multiple sensor data; A plurality of encoders are used to respectively perform feature extraction on the sensor data to obtain the plurality of target features.
3. The method according to claim 2, characterized in that The method further comprises: using a first sensor data from among the plurality of sensor data to obtain weather characteristics and performing weight adaptive calculation based on the weather characteristics to obtain weights corresponding to each target characteristic, and further comprising: using a first encoder to perform feature extraction on the first sensor data to obtain the weather characteristics and using a weight adaptive extraction network to calculate the weights corresponding to each target characteristic based on the weather characteristics; A weighted fusion feature is obtained based on the multiple target features and the corresponding weights, and target perception and recognition is performed based on the weighted fusion feature, further including: multiplying each target feature by the corresponding weight and then fusing them to obtain the weighted fusion feature, and using a perception model to perform target perception and recognition based on the weighted fusion feature.
4. The method according to claim 3, characterized in that Also includes: The plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model are trained.
5. The method according to claim 4, characterized in that Training the plurality of encoders, the first encoder, the weight adaptive extraction network, and the perception model further includes: Constructing a first training set and a second training set, wherein the first training set includes the multiple sensor data, and the second training set includes the first sensor data; Using the first training set to train the plurality of encoders and the perception model, and using the second training set to train the first encoder and the weight adaptive extraction network; Based on the first loss function and the second loss function, the multiple encoders, the first encoder, the weight adaptive extraction network, the perception model, and the weather recognition network are iteratively optimized until the first loss function and the second loss function meet preset requirements, and the training is stopped.
6. The method according to claim 4, characterized in that Training the first encoder and the weight adaptive extraction network further includes: constructing a third training set, wherein the third training set includes the first sensor data; Using the third training set to train the first encoder and the weight adaptive extraction network; The first encoder, the weight adaptive extraction network and the weather recognition network are iteratively optimized based on the third loss function until the third loss function reaches a preset requirement and the training is stopped.
7. The method according to claim 6, characterized in that Training the plurality of encoders and the perception model further comprises: constructing a fourth training set, wherein the fourth training set includes the multiple sensor data; Using the fourth training set to train the plurality of encoders and the perception model; The plurality of encoders and the perception model are iteratively optimized based on the fourth loss function until the fourth loss function meets a preset requirement and the training is stopped.
8. A target perception and recognition system, characterized in that: include: a feature extraction module configured to obtain a plurality of sensor data of a target and extract a plurality of target features from the plurality of sensor data; A calculation module, configured to obtain weather characteristics using first sensor data among the plurality of sensor data and to perform weight adaptive calculation based on the weather characteristics to obtain a weight corresponding to each target characteristic; The recognition module is configured to obtain a weighted fusion feature based on the multiple target features and the corresponding weights and perform target perception recognition based on the weighted fusion feature.
9. A computer device comprising: at least one processor; as well as A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method according to any one of claims 1 to 7 when executing the program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.