Traffic accident detection method, system, equipment and medium

By conducting unsupervised training on the radar encoder and combining text feature recognition, the problems of limited computing resources and inaccurate manual labeling in existing traffic accident detection are solved, which improves the accuracy and efficiency of accident recognition and reduces system costs.

CN119202805AInactive Publication Date: 2024-12-27TIANYI TRANSPORTATION TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411718750.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-27
Publication Date
2024-12-27
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Due to limited computing resources and inaccurate manual labeling, the existing traffic accident detection schemes have slow data processing speed and poor accuracy, which increases system costs and increases failure risk, and are poor in robustness and accuracy.

Method used

By acquiring paired image data and radar data, unsupervised training of the radar encoder, using the optimized radar encoder for traffic accident recognition, combining text features and radar features for similarity calculation, automatically determining the type of accident, reducing artificial Dependency for labeling.

Benefits of technology

It improves the accuracy of traffic accident identification, saves manpower and time costs, optimizes the processing process, reduces errors caused by manual labeling, and improves the efficiency of the entire accident identification process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119202805A_ABST
    Figure CN119202805A_ABST
Patent Text Reader

Abstract

The invention discloses a traffic accident detection method, system and device and a medium. The method comprises the following steps: acquiring paired image data and radar data; encoding the image data to obtain image features, and encoding the radar data by using a radar encoder to obtain radar features; optimizing the radar encoder based on the paired image features and radar features to obtain an optimized radar encoder; encoding the collected radar data by using the optimized radar encoder to obtain radar features; and performing traffic accident identification based on the radar features to obtain an accident identification result of the collected radar data. According to the scheme provided by the invention, the radar encoder is subjected to unsupervised training by using massive paired image data and radar data, so that the trained radar encoder code is more beneficial to subsequent accident identification, and the identification accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vehicle-road collaboration, and in particular to a traffic accident detection method, system, equipment and medium. Background Art

[0002] At present, common traffic accident detection mainly relies on road perception equipment (such as cameras, radars, etc.) to collect relevant perception data. Then, neural network technology is used to perceive and identify these collected data to distinguish different types of traffic accidents, such as rear-end collisions, scratches, collisions with pedestrians, etc.

[0003] Since the amount of perception data collected by the perception device may be relatively large, for example, high-resolution images will contain a large amount of pixel information, a large amount of data will take up a lot of storage space in subsequent processing and the transmission will also consume time and other resources. Processing these large-scale perception data, especially high-resolution images, requires powerful computing power to support the operation of neural networks. However, the actual situation is often that computing resources are limited and cannot meet the needs well. This leads to a slowdown in the analysis and processing of data, and may even cause freezes, which ultimately makes the actual effect of traffic accident classification unsatisfactory and unable to accurately and quickly judge the type of accident. In addition, in order to process these data to improve computing power, the cost of the system will increase, including the cost of hardware purchase, operation and maintenance, and it will also make the structure of the system more complicated, increasing the possibility of system failures and other problems.

[0004] In addition, in the process of training and detecting with neural networks, it is usually necessary to manually annotate the collected perception data, such as annotating the specific location of the accident in the image, the vehicles or pedestrians involved, and other information. However, due to subjective judgment differences and fatigue in manual operation, the annotation is not accurate enough. This inaccurate manual annotation will cause the neural network to receive wrong information during the learning and recognition process, making it difficult to accurately detect some key information related to traffic accidents, such as the exact location of the accident, the actual objects involved, etc. Ultimately, it affects the accuracy of the entire detection scheme, making it impossible to reliably classify and judge traffic accidents.

[0005] Taking into account the above-mentioned problems in data processing and manual labeling, the existing traffic accident detection schemes have poor robustness and accuracy when facing actual situations, and are easily affected by factors such as data and labeling, resulting in inaccurate detection results. Summary of the invention

[0006] In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present invention provides a traffic accident detection method, comprising the following steps: Acquire paired image data and radar data; Encoding the image data to obtain image features and encoding the radar data using a radar encoder to obtain radar features; Optimizing the radar encoder based on the paired image features and radar features to obtain an optimized radar encoder; Encoding the collected radar data using the optimized radar encoder to obtain radar features; Traffic accident identification is performed based on the radar features to obtain an accident identification result of the collected radar data.

[0007] In some embodiments, traffic accident identification based on the radar features further includes: Acquire multiple constructed texts and encode each text to obtain text features; Calculating the similarity between the radar feature and each of the text features; The text corresponding to the collected radar data is determined according to the similarity, and the type of traffic accident is determined based on the corresponding text.

[0008] In some embodiments, obtaining the constructed multiple texts and encoding each text to obtain text features further includes: Construct multiple pairs of images and image texts; Encoding the image using an image encoder and encoding the image text using a text encoder; Based on the fact that the similarity between the features obtained after encoding the paired image and the image text is 1, and the similarity between the features obtained after encoding the unpaired image and the image text is 0, the image encoder and the text encoder are optimized; In response to the image encoder and the text encoder reaching a preset requirement, stopping the optimization; The constructed multiple texts are encoded using the optimized text encoder.

[0009] In some embodiments, the radar encoder is optimized based on the paired image features and radar features to obtain the optimized radar encoder, further comprising: The radar encoder is optimized based on the similarity between the features obtained after the paired image data and the radar data are encoded being 1, and the similarity between the features obtained after the unpaired image data and the radar data are encoded being 0; In response to the radar encoder meeting the preset requirement, the optimization is stopped.

[0010] In some embodiments, traffic accident identification based on the radar features further includes: The radar features are classified and identified using an accident identification network to obtain the traffic accident type of the collected radar data.

[0011] In some embodiments, classifying and identifying the radar features using an accident identification network to obtain the traffic accident type of the collected radar data further includes: Acquire radar data corresponding to the traffic accident and use the optimized radar encoder to encode the radar data corresponding to the traffic accident to obtain a radar feature training set; Training the accident recognition network using the radar feature training set; The accident recognition network is iteratively optimized based on the loss function until the loss function reaches a preset requirement and the training is stopped.

[0012] In some embodiments, it also includes: In response to receiving an instruction to retrieve radar data corresponding to a preset text, similarities between a text feature corresponding to the preset text and a plurality of radar features are calculated and corresponding radar data are returned according to the similarities.

[0013] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a traffic accident detection system, comprising: an acquisition module configured to acquire paired image data and radar data; A first encoding module is configured to encode the image data to obtain image features and to encode the radar data using a radar encoder to obtain radar features; an optimization module, configured to optimize the radar encoder based on the paired image features and the radar features to obtain the optimized radar encoder; A second encoding module is configured to encode the collected radar data using the optimized radar encoder to obtain radar features; The identification module is configured to perform traffic accident identification based on the radar features to obtain an accident identification result of the collected radar data.

[0014] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer device, including: at least one processor; and A memory storing a computer program executable on the processor, wherein the processor executes the steps of any one of the traffic accident detection methods described above when executing the program.

[0015] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the traffic accident detection methods described above are performed.

[0016] The present invention has one of the following beneficial technical effects: The solution proposed in the present invention uses a large amount of paired image data and radar data to carry out unsupervised training on the radar encoder. Unsupervised training means that there is no need to manually label the data with information such as categories in advance, and the model can learn features from the inherent structure of the data. The code generated by the radar encoder optimized in this way is more beneficial for subsequent accident identification. Because the encoder that has undergone unsupervised training on a large amount of data can better capture features that are helpful for accident identification, the accuracy of accident identification is improved. Moreover, since there is no need for manual labeling, a lot of manpower and time costs are saved, further improving the efficiency of the entire accident identification process. While improving the accident identification effect, the processing flow is also optimized, reducing the errors that may be caused by manual labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a traffic accident detection method provided by an embodiment of the present invention; Figure 2 A schematic diagram of matching radar features with text features provided by an embodiment of the present invention; Figure 3 A schematic diagram of training image features and text features provided by an embodiment of the present invention; Figure 4 A schematic diagram of training image features and radar features provided by an embodiment of the present invention; Figure 5 A schematic diagram of an accident identification network provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of a traffic accident detection system provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention; Figure 8A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0020] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0021] In an embodiment of the present invention, the radar may be a laser radar or a millimeter wave radar, such as a 3D millimeter wave radar, a 4D millimeter wave radar, and the like.

[0022] According to one aspect of the present invention, an embodiment of the present invention provides a traffic accident detection method, such as Figure 1 As shown, it may include the steps of: S1, obtain paired image data and radar data; S2, encoding the image data to obtain image features and using a radar encoder to encode the radar data to obtain radar features; S3, optimizing the radar encoder based on the paired image features and the radar features to obtain an optimized radar encoder; S4, using the optimized radar encoder to encode the collected radar data to obtain radar features; S5, performing traffic accident identification based on the radar features to obtain an accident identification result of the collected radar data.

[0023] In some embodiments, radar sensors and image sensors can be simultaneously installed on the roadside. This arrangement makes it relatively easy to obtain paired image data and radar data. These two sensors can simultaneously observe and record road scenes, thereby providing paired image data and radar data. In addition, when performing unsupervised training on the radar encoder, since the Internet integrates data from multiple different sources (such as other related monitoring systems, shared data platforms, etc.), the Internet can be used to easily construct paired image and radar data, further enriching the source of data pairs. In this way, the solution proposed by the present invention can make the trained radar encoder code more conducive to subsequent accident identification by using massive paired image data and radar data to perform unsupervised training on the radar encoder, thereby improving the recognition accuracy, and at the same time, no manual labeling is required, further improving efficiency.

[0024] In some embodiments, traffic accident identification based on the radar features further includes: Acquire multiple constructed texts and encode each text to obtain text features; Calculating the similarity between the radar feature and each of the text features; The text corresponding to the collected radar data is determined according to the similarity, and the type of traffic accident is determined based on the corresponding text.

[0025] Specifically, Figure 2 As shown in Fig. 1, a text set including multiple texts can be constructed, such as A photo of a [traffic jam]. / A photo of a [car-car accident]… / A photo of a [normal], and the text features T1~T1 can be obtained by encoding them using the text encoder. N After using the optimized radar encoder to encode the radar data collected in real time to obtain the radar feature R1, the similarity between the radar feature and each text feature, such as cosine similarity, can be calculated. By sorting by similarity, the text corresponding to the maximum similarity is found, and the type of traffic accident corresponding to the real-time collected radar data can be determined based on the text.

[0026] In some embodiments, CLIP (Contrastive Language-Image Pre-training) can be used to encode images and texts, such as Figure 3 As shown in the figure, in CLIP, images and corresponding text descriptions are used as positive sample pairs, and different images and text descriptions are used as negative sample pairs. The goal of the model is to make the feature representations of positive sample pairs as close as possible and the feature representations of negative sample pairs as far apart as possible in the feature space, so as to learn the semantic association between images and texts. In this way, image and text alignment can be achieved in CLIP. The radar encoder can be trained using the image features encoded by CLIP to align the image and radar, and then indirectly align the radar and text. Finally, the type of traffic accident can be identified by calculating the similarity between radar features and text features.

[0027] In some embodiments, obtaining the constructed multiple texts and encoding each text to obtain text features further includes: Construct multiple pairs of images and image texts; Encoding the image using an image encoder and encoding the image text using a text encoder; Based on the fact that the similarity between the features obtained after encoding the paired image and the image text is 1, and the similarity between the features obtained after encoding the unpaired image and the image text is 0, the image encoder and the text encoder are optimized; In response to the image encoder and the text encoder reaching a preset requirement, stopping the optimization; The constructed multiple texts are encoded using the optimized text encoder.

[0028] Specifically, we can use Internet images and their corresponding text descriptions to construct a training set D = {Ii, Ti}, where i = 1…N, representing N <image, text> pairs in a training batch, where Ii represents the i-th image and Ti represents the i-th text. I and E T They represent image encoder and text encoder, which are used to encode images and text respectively. The similarity between the features obtained after encoding paired images and image texts is 1, indicating that they are highly correlated. The similarity between the features obtained after encoding unpaired images and image texts is 0, indicating that they are not correlated, that is: E I (I i )·E T (T i )=1,E I (I i ) •E T (T j )=0, i≠j, the image encoder and the text encoder are optimized, and finally the optimized image encoder is used for image encoding and the optimized text encoder is used for text encoding.

[0029] In some embodiments, the radar encoder is optimized based on the paired image features and radar features to obtain the optimized radar encoder, further comprising: The radar encoder is optimized based on the similarity between the features obtained after the paired image data and the radar data are encoded being 1, and the similarity between the features obtained after the unpaired image data and the radar data are encoded being 0; In response to the radar encoder meeting the preset requirement, the optimization is stopped.

[0030] Specifically, Figure 4 As shown, the image encoder can be fixed, the image-radar dataset with the same viewing angle and time synchronization can be collected, and the radar encoder E can be trained unsupervised. R (R i), which can indirectly align radar data with text. In addition, during the radar encoder learning process, the BEV method can be used to process radar data. The training target is that the features of paired image-radar data samples are close; the features of unpaired image-radar data samples are far apart, that is: E I (I i ) •E R (R i )=1,E I (I i ) •E R (R j )=0, i≠j, that is, paired image-radar data samples, so that E R (R i ) output and E I (I i ) output correlation coefficient close to 1; unpaired image-radar data samples, making E R (R j ) output and E I (I i ) is close to 0. By using such an objective function, the radar encoder E is optimized. R In this way, by fixing the image encoder and only training the radar encoder, the radar data can be indirectly aligned with the text, thereby achieving zero-sample recognition of traffic accident types.

[0031] In some embodiments, traffic accident identification based on the radar features further includes: The radar features are classified and identified using an accident identification network to obtain the traffic accident type of the collected radar data.

[0032] Specifically, when classifying and identifying traffic accidents, the coding features output by the radar encoder can be input into the accident identification network, and the classification and identification can be performed through the accident identification network, such as Figure 5 As shown, the accident recognition network Cls can be based on a convolutional neural network similar to ResNet, where x is E R (R) is the output, c is the classification result, E R The recognition accuracy of the accident recognition network Cls can also be improved by continuously optimizing the network parameters of Cls.

[0033] In some embodiments, classifying and identifying the radar features using an accident identification network to obtain the traffic accident type of the collected radar data further includes: Acquire radar data corresponding to the traffic accident and use the optimized radar encoder to encode the radar data corresponding to the traffic accident to obtain a radar feature training set; Training the accident recognition network using the radar feature training set; The accident recognition network is iteratively optimized based on the loss function until the loss function reaches a preset requirement and the training is stopped.

[0034] Specifically, since the radar encoder has been optimized through massive image-radar data, a small amount of radar data corresponding to traffic accidents can be used as training samples to optimize the accident recognition network and obtain better recognition results.

[0035] During training, we can use the loss function Loss (GT, Cls(E R (R))), such as cross entropy loss, and trained using the gradient backpropagation method, where GT represents ground truth, i.e., the true value of the traffic accident type, and E R is a fixed radar encoder and Cls is a traffic accident recognition network.

[0036] In some embodiments, it also includes: In response to receiving an instruction to retrieve radar data corresponding to a preset text, similarities between a text feature corresponding to the preset text and a plurality of radar features are calculated and corresponding radar data are returned according to the similarities.

[0037] Specifically, since radar features are indirectly aligned with text features, similar radar data can be quickly retrieved using text. R , encode all radar data in the data set to be retrieved to obtain the radar feature set, and then use the text encoder E T Encode a given specific text (e.g., A photo of a collision with many cars) to obtain the text feature F T , using text feature F T Find similar radar features from the radar coding feature set, sort them according to similarity, and output the corresponding radar data.

[0038] Since radar sensors and image sensors exist on the roadside at the same time, it is relatively easy to obtain paired image data and radar data, and paired image data and radar data can also be easily constructed using the Internet. The solution proposed by the present invention uses massive paired image data and radar data to perform unsupervised training on the radar encoder, so that the trained radar encoder encoding is more conducive to subsequent accident identification, thereby improving the recognition accuracy, and no manual labeling is required, further improving the efficiency.

[0039] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a traffic accident detection system 400, such as Figure 6 As shown, including: An acquisition module 401 is configured to acquire paired image data and radar data; A first encoding module 402 is configured to encode the image data to obtain image features and use a radar encoder to encode the radar data to obtain radar features; An optimization module 403 is configured to optimize the radar encoder based on the paired image features and the radar features to obtain the optimized radar encoder; A second encoding module 404 is configured to encode the collected radar data using the optimized radar encoder to obtain radar features; The identification module 405 is configured to perform traffic accident identification based on the radar features to obtain an accident identification result of the collected radar data.

[0040] In some embodiments, traffic accident identification based on the radar features further includes: Acquire multiple constructed texts and encode each text to obtain text features; Calculating the similarity between the radar feature and each of the text features; The text corresponding to the collected radar data is determined according to the similarity, and the type of traffic accident is determined based on the corresponding text.

[0041] In some embodiments, obtaining the constructed multiple texts and encoding each text to obtain text features further includes: Construct multiple pairs of images and image texts; Encoding the image using an image encoder and encoding the image text using a text encoder; Based on the fact that the similarity between the features obtained after encoding the paired image and the image text is 1, and the similarity between the features obtained after encoding the unpaired image and the image text is 0, the image encoder and the text encoder are optimized; In response to the image encoder and the text encoder reaching a preset requirement, stopping the optimization; The constructed multiple texts are encoded using the optimized text encoder.

[0042] In some embodiments, the radar encoder is optimized based on the paired image features and radar features to obtain the optimized radar encoder, further comprising: The radar encoder is optimized based on the similarity between the features obtained after the paired image data and the radar data are encoded being 1, and the similarity between the features obtained after the unpaired image data and the radar data are encoded being 0; In response to the radar encoder meeting the preset requirement, the optimization is stopped.

[0043] In some embodiments, traffic accident identification based on the radar features further includes: The radar features are classified and identified using an accident identification network to obtain the traffic accident type of the collected radar data.

[0044] In some embodiments, classifying and identifying the radar features using an accident identification network to obtain the traffic accident type of the collected radar data further includes: Acquire radar data corresponding to the traffic accident and use the optimized radar encoder to encode the radar data corresponding to the traffic accident to obtain a radar feature training set; Training the accident recognition network using the radar feature training set; The accident recognition network is iteratively optimized based on the loss function until the loss function reaches a preset requirement and the training is stopped.

[0045] In some embodiments, it also includes: In response to receiving an instruction to retrieve radar data corresponding to a preset text, similarities between a text feature corresponding to the preset text and a plurality of radar features are calculated and corresponding radar data are returned according to the similarities.

[0046] Based on the same inventive concept, according to another aspect of the present invention, Figure 7 As shown, an embodiment of the present invention further provides a computer device 501, including: at least one processor 520; and The memory 510 stores a computer program 511 that can be run on the processor. When the processor 520 executes the program, the steps of any of the above traffic accident detection methods are performed.

[0047] Based on the same inventive concept, according to another aspect of the present invention, Figure 8 As shown, an embodiment of the present invention further provides a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, the steps of any of the above traffic accident detection methods are performed.

[0048] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0049] Furthermore, it should be appreciated that the computer-readable storage medium (eg, memory) herein may be either volatile memory or nonvolatile memory, or may include both volatile memory and nonvolatile memory.

[0050] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0051] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0052] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0053] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0054] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0055] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A traffic accident detection method, characterized in that: The following steps are involved: Acquire paired image data and radar data; Encoding the image data to obtain image features and encoding the radar data using a radar encoder to obtain radar features; Optimizing the radar encoder based on the paired image features and radar features to obtain an optimized radar encoder; Encoding the collected radar data using the optimized radar encoder to obtain radar features; Traffic accident identification is performed based on the radar features to obtain an accident identification result of the collected radar data.

2. The method according to claim 1, characterized in that The method further comprises: identifying a traffic accident based on the radar feature; Acquire multiple constructed texts and encode each text to obtain text features; Calculating the similarity between the radar feature and each of the text features; The text corresponding to the collected radar data is determined according to the similarity, and the type of traffic accident is determined based on the corresponding text.

3. The method according to claim 2, characterized in that Acquiring multiple constructed texts and encoding each text to obtain text features further includes: Construct multiple pairs of images and image texts; Encoding the image using an image encoder and encoding the image text using a text encoder; Based on the fact that the similarity between the features obtained after encoding the paired image and the image text is 1, and the similarity between the features obtained after encoding the unpaired image and the image text is 0, the image encoder and the text encoder are optimized; In response to the image encoder and the text encoder reaching a preset requirement, stopping the optimization; The constructed multiple texts are encoded using the optimized text encoder.

4. The method according to claim 1, characterized in that The radar encoder is optimized based on the paired image features and radar features to obtain the optimized radar encoder, further comprising: The radar encoder is optimized based on the similarity between the features obtained after the paired image data and the radar data are encoded being 1, and the similarity between the features obtained after the unpaired image data and the radar data are encoded being 0; In response to the radar encoder meeting the preset requirement, the optimization is stopped.

5. The method according to claim 1, characterized in that The method further comprises: identifying a traffic accident based on the radar feature; The radar features are classified and identified using an accident identification network to obtain the traffic accident type of the collected radar data.

6. The method according to claim 5, characterized in that Using an accident recognition network to classify and identify the radar features to obtain the traffic accident type of the collected radar data further includes: Acquire radar data corresponding to the traffic accident and use the optimized radar encoder to encode the radar data corresponding to the traffic accident to obtain a radar feature training set; Training the accident recognition network using the radar feature training set; The accident recognition network is iteratively optimized based on the loss function until the loss function reaches a preset requirement and the training is stopped.

7. The method according to claim 1, characterized in that Also includes: In response to receiving an instruction to retrieve radar data corresponding to a preset text, similarities between a text feature corresponding to the preset text and a plurality of radar features are calculated and corresponding radar data are returned according to the similarities.

8. A traffic accident detection system, characterized in that: include: an acquisition module configured to acquire paired image data and radar data; A first encoding module is configured to encode the image data to obtain image features and to encode the radar data using a radar encoder to obtain radar features; an optimization module, configured to optimize the radar encoder based on the paired image features and the radar features to obtain the optimized radar encoder; A second encoding module is configured to encode the collected radar data using the optimized radar encoder to obtain radar features; The identification module is configured to perform traffic accident identification based on the radar features to obtain an accident identification result of the collected radar data.

9. A computer device comprising: at least one processor; as well as A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

Citation Information

Patent Citations

  • Target identification method and device, electronic equipment and storage medium

    CN115578707A

  • Depth estimation method based on fusion of visual image and millimeter wave radar feature matrix

    CN116363186A

  • Remote sensing image cross-modal retrieval method based on language and visual detail feature fusion

    CN116775922A

  • Computationally efficient unsupervised DNN pre-training

    CN117521748A

  • Target detection method and device

    CN117994494A