Traffic accident detection method, device, equipment and medium

Abstract

The invention relates to the technical field of computers, and provides a traffic accident detection method, device and equipment and a medium, and the method comprises the steps: marking each object in a plurality of traffic accident images; calculating correlation between image features in the traffic accident image and a plurality of traffic accident text features based on an object detection model, and forming a feature map based on the correlation; predicting each object in the traffic accident image based on the feature map, updating the object detection model based on a prediction result and the mark, and returning to the step of calculating the correlation until a preset condition is reached; and carrying out traffic accident detection through the trained object detection model. According to the scheme, decoupling of related factors of the traffic accident is realized, and whether the traffic accident occurs is indirectly judged by monitoring the object related to the traffic accident in the image, so that the accuracy of traffic accident detection is improved, the demand on a traffic accident training sample is greatly reduced, and the traffic accident detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a traffic accident detection method, device, equipment and medium. Background Art

[0002] Traffic accident detection is an essential part of ensuring road traffic safety. By detecting road traffic accidents, timely warnings of potential traffic accidents can be issued to improve road traffic safety.

[0003] In the existing traffic accident detection methods, on the one hand, by observing whether the target object has abnormal changes in multiple consecutive frames of images to determine whether a traffic accident has occurred, different target objects will often produce different judgment results, resulting in low accuracy in traffic accident detection. On the other hand, through model learning to identify the target area that needs to be monitored in the image, monitoring the abnormal situation of the target area to detect whether a traffic accident has occurred, a large amount of traffic accident data is required to improve the accuracy of identifying the target area. In actual traffic scenes, the probability of traffic accidents is low, and it is difficult to form a large-scale training set for traffic accident scenes. In addition, there are large deviations between images captured at different locations or devices, making it difficult to use them directly for model training.

[0004] Based on this, there is an urgent need for a traffic accident detection method that has low dependence on traffic accident scenes and can improve the efficiency and accuracy of traffic accident detection. Summary of the invention

[0005] In view of this, based on the problems that traffic accident detection in the prior art has low accuracy and high dependence on traffic accident scenes, in actual traffic scenes, the probability of traffic accidents is low, it is difficult to form a large-scale training set for traffic accident scenes, and there is a large deviation between images captured at different locations or devices, which makes it difficult to use them directly for model training. The present invention proposes a traffic accident detection method, device, equipment and medium.

[0006] Based on the above objectives, one aspect of the present invention provides a traffic accident detection method, comprising: Marking each object in a plurality of traffic accident images; Calculating the correlation between the image features in the traffic accident image and a plurality of traffic accident text features based on the object detection model, and forming a feature map based on the correlation; Predicting each object in the traffic accident image based on the feature map, updating the object detection model based on the prediction result and the label, and returning to the step of calculating the correlation until a preset condition is met; Traffic accident detection using trained object detection models.

[0007] In some embodiments, the step of calculating the correlation between the image feature in the traffic accident image and a plurality of traffic accident text features includes: Mapping the image features and the plurality of traffic accident text features in the traffic accident image to a unified feature space to convert them into vectors of the same dimension; The vector distance values ​​respectively corresponding to the image features and the plurality of traffic accident text features in the unified feature space are calculated to determine the correlation.

[0008] In some embodiments, the step of forming a feature map based on the correlation comprises: Determine the minimum vector distance value among the calculated distance values; Determining the text feature with the greatest correlation among the plurality of traffic accident text features based on the minimum vector distance value; The text feature with the greatest correlation is fused with the image feature based on an aggregation algorithm to form a feature map.

[0009] In some embodiments, the step of predicting each object in the traffic accident image based on the feature map, updating the object detection model based on the prediction result and the label, and returning to the step of calculating the correlation until a preset condition is met includes: Predicting the category and bounding box of each object in the traffic accident image based on the feature map and locating the corresponding area of ​​each object in the traffic accident image based on the category and bounding box of each object in the traffic accident image; Obtaining the correlation of the feature graph corresponding to the corresponding area, and calculating the correlation confidence corresponding to each object based on the correlation; The object detection model is updated based on the correlation confidence and the label and the step of calculating the correlation is returned until the predicted category and bounding box corresponding to each object approximate the corresponding label value and the correlation confidence reaches a correlation confidence threshold.

[0010] In some embodiments, the step of performing traffic accident detection using the trained object detection model includes: Based on the current frame image and the text related to the traffic accident, an object matching the text related to the traffic accident in the current frame image is obtained through a trained object detection model, and a category and a bounding box of the matched object and a confidence level of the corresponding image-text correlation are determined; A plurality of continuous frame images after the current frame image are monitored, and traffic accident detection is performed based on the monitoring results of the plurality of continuous frame images.

[0011] In some embodiments, the step of monitoring a plurality of consecutive image frames after the current image frame and performing traffic accident detection based on the monitoring results of the plurality of consecutive image frames includes: Obtaining objects that match the text in a plurality of consecutive frame images after the current frame image through the trained object detection model, respectively determining and accumulating the confidences of the corresponding image-text correlations to obtain a confidence accumulation value; If the accumulated confidence value exceeds the confidence threshold of the traffic accident occurrence, it is confirmed that a traffic accident has occurred.

[0012] In some embodiments, the method further comprises: Extracting image features from the traffic accident image based on an image encoder, which includes dividing the traffic accident image into a plurality of image blocks and extracting image features from each image block; Several traffic accident text features are extracted from the traffic accident text set based on the text encoder.

[0013] Another aspect of the embodiment of the present invention further provides a traffic accident detection device, including: The first module is used to mark each object in a plurality of traffic accident images; A second module is used to calculate the correlation between the image features in the traffic accident image and a plurality of traffic accident text features based on the object detection model, and form a feature map based on the correlation; A third module is used to predict each object in the traffic accident image based on the feature map, update the object detection model based on the prediction result and the mark, and return to the step of calculating the correlation until a preset condition is met; The fourth module is used to detect traffic accidents through the trained object detection model.

[0014] Another aspect of an embodiment of the present invention provides an electronic device, comprising at least one processor; and a memory, wherein the memory stores computer instructions that can be executed on the processor, and when the instructions are executed by the processor, the steps of the above-mentioned traffic accident detection method are implemented.

[0015] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned traffic accident detection method are implemented.

[0016] The present invention has at least the following beneficial effects: by marking each object in a plurality of traffic accident images, calculating the correlation between the image features in the traffic accident images and a number of traffic accident text features based on the object detection model and forming a feature map based on the correlation, predicting each object in the traffic accident image based on the feature map, updating the object detection model based on the prediction result and the marking, and then returning to the step of calculating the correlation until the preset conditions are met, and performing traffic accident detection through the trained object detection model. The above-mentioned traffic accident detection method realizes the decoupling of factors related to traffic accidents, indirectly determines whether a traffic accident has occurred by monitoring objects related to traffic accidents in the image, and aligns with the cognition of the occurrence of actual traffic accidents, thereby improving the accuracy of traffic accident detection. At the same time, there is no need to analyze the overall image data in continuous time, and only the objects related to traffic accidents in the image need to be analyzed, which greatly reduces the demand for traffic accident training samples and is conducive to improving the efficiency of traffic accident detection.

[0017] In addition, the traffic accident detection device, electronic device and medium of the present invention also have at least the above beneficial effects, which will not be elaborated. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flow chart of a traffic accident detection method provided by the first embodiment of the present invention; Figure 2 A schematic diagram showing the process of training an object detection model provided by a second embodiment of the present invention is shown; Figure 3 A schematic diagram of traffic accident reasoning based on a trained object detection model provided by a second embodiment of the present invention is shown; Figure 4 A schematic structural diagram of a traffic accident detection device provided by a third embodiment of the present invention; Figure 5 A schematic structural diagram of an electronic device provided in accordance with a fourth embodiment of the present invention; Figure 6 A schematic structural diagram of a computer-readable storage medium provided for the fifth embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following describes embodiments of the present invention. However, it should be understood that the disclosed embodiments are merely examples and other embodiments can take various alternative forms.

[0021] Furthermore, it should be noted that the terms "includes," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but may also include elements not explicitly listed or inherent to these processes, methods, articles, or apparatuses.

[0022] One or more embodiments of the present application will be described below with reference to the accompanying drawings.

[0023] Based on the above objectives, a first aspect of an embodiment of the present invention provides an embodiment of a traffic accident detection method. Figure 1 FIG. 1 is a flow chart of a traffic accident detection method provided by an embodiment of the present invention, such as Figure 1 As shown, including: S1. marking each object in a plurality of traffic accident images; S2. Calculate the correlation between the image features in the traffic accident image and the features of the plurality of traffic accident texts based on the object detection model, and form a feature map based on the correlation; S3, predicting each object in the traffic accident image based on the feature map, updating the object detection model based on the prediction results and the labels, and returning to the step of calculating the correlation until a preset condition is met; S4. Perform traffic accident detection using the trained object detection model.

[0024] The above-mentioned traffic accident detection method realizes the decoupling of factors related to traffic accidents. By detecting objects related to traffic accidents in the image, it indirectly determines whether a traffic accident has occurred, and is aligned with the cognition of the actual occurrence of traffic accidents, thereby improving the accuracy of traffic accident detection. At the same time, there is no need to analyze the overall image data in continuous time, but only to analyze the objects related to traffic accidents in the image, which greatly reduces the demand for traffic accident training samples and is conducive to improving the efficiency of traffic accident detection.

[0025] According to some embodiments of the present invention, the step of calculating the correlation between the image features in the traffic accident image and the several traffic accident text features includes: mapping the image features in the traffic accident image and the several traffic accident text features to a unified feature space to convert them into vectors of the same dimension; calculating the vector distance values ​​corresponding to the image features and the several traffic accident text features in the unified feature space to determine the correlation. Converting the image features and the text features into vectors of the same dimension and calculating the correlation realizes cross-modal data fusion, which is conducive to a more comprehensive reflection of traffic accidents.

[0026] According to several embodiments of the present invention, the step of forming a feature map based on correlation includes: determining the minimum vector distance value among several distance values ​​obtained by calculation; determining the text feature with the greatest correlation among several text features of traffic accidents based on the minimum vector distance value; and fusing the text feature with the greatest correlation with the image feature based on an aggregation algorithm to form a feature map. Based on the principle that the larger the vector distance value, the smaller the corresponding correlation, the minimum vector distance value is determined among several distance values ​​obtained by calculation to accurately locate the text feature with the greatest correlation with the image feature, which is conducive to improving the efficiency and accuracy of traffic accident analysis.

[0027] According to some embodiments of the present invention, each object in the traffic accident image is predicted based on the feature map, the object detection model is updated based on the prediction result and the mark, and the step of calculating the correlation is returned until the preset condition is reached, including: predicting the category and bounding box of each object in the traffic accident image based on the feature map and locating the corresponding area of ​​each object in the traffic accident image based on the two; obtaining the correlation of the feature map corresponding to the corresponding area, and calculating the correlation confidence corresponding to each object based on it; updating the object detection model based on the correlation confidence and the mark and returning to the step of calculating the correlation until the predicted category and bounding box of each object are close to the corresponding mark value, and the correlation confidence reaches the correlation confidence threshold. The object detection model is continuously optimized by the loss function and the correlation confidence reaches the correlation confidence threshold to improve the accuracy of the object detection model in locating objects in the image.

[0028] According to some embodiments of the present invention, the step of performing traffic accident detection through a trained object detection model includes: based on the current frame image and the text related to the traffic accident, obtaining the object matching the text related to the traffic accident in the current frame image through the trained object detection model, and determining the category and bounding box of the matched object and the confidence of the corresponding image-text correlation; monitoring a number of consecutive frame images after the current frame image, and performing traffic accident detection based on the monitoring results of the consecutive frame images. Using the trained object detection model to identify sensitive objects related to traffic accidents in real time, determining the category and bounding box of the object can accurately locate the key elements of the accident scene, and monitoring a number of consecutive frame images after the current frame image is helpful for early warning before a traffic accident occurs.

[0029] According to some embodiments of the present invention, the steps of monitoring a plurality of consecutive frames after the current frame image and detecting a traffic accident based on the monitoring results of the plurality of consecutive frames image include: obtaining objects respectively matching the traffic accident text in a plurality of consecutive frames image after the current frame image through the trained object detection model, respectively determining the confidence of the corresponding image text correlation and accumulating them to obtain the confidence accumulation value; if the confidence accumulation value exceeds the confidence threshold of the traffic accident occurrence, confirming the occurrence of the traffic accident. By accumulating the confidence of the matching objects in the consecutive frames, the continuous correlation between the objects and the traffic accident text is reflected, which is conducive to improving the robustness.

[0030] According to several embodiments of the present invention, the method also includes: extracting image features in the traffic accident image based on an image encoder, which includes dividing the traffic accident image into several image blocks and extracting image features in each image block; extracting several traffic accident text features in the traffic accident text set based on a text encoder.

[0031] In order to further understand a traffic accident detection method of the present invention, a specific example is described in detail below. The traffic accident detection at a crossroad is used as an example for explanation.

[0032] like Figure 2 As shown, Figure 2 The process diagram of training the object detection model provided by the second embodiment of the present invention is shown. In the data preparation stage, the roadside equipment installed on both sides of the road collects road traffic images in real time, that is, frames of images, and marks each object in the collected multiple traffic accident images, and marks the category and bounding box of each object, which is used for comparison of the corresponding categories and bounding boxes predicted in the subsequent training of the object detection model, so as to improve the accuracy of the object detection model in locating the object. At the same time, a traffic accident text set related to the traffic accident is obtained, including the names of traffic participants (such as large trucks, cars, bicycles, pedestrians, etc.) and the names of scattered objects (such as glass fragments, paint fragments, triangular cones, etc.).

[0033] In the data preprocessing stage, the labeled traffic accident images and traffic accident text sets are preprocessed respectively to extract the corresponding image features and text features. Specifically, the traffic accident images are input into a pre-trained image encoder, and the image encoder is used to process the images and extract multi-scale image features. For example, the backbone network of the YOLO (You Only Look Once, real-time target detection algorithm) series network (trained by deep learning and optimization algorithms) is used to extract rich image features in traffic accident images; or the backbone network of the DETR (Detection Transformer, end-to-end target detection network) network based on the Transformer encoder-decoder architecture (pre-trained neural network) is used to extract rich image features in traffic accident images. The traffic accident text set is input into a pre-trained text encoder (such as a Transformer-based encoder), and the text encoder is used to process and extract text features related to traffic accidents.

[0034] In the training object detection model stage, the image features extracted from multiple traffic accident images are input into the object detection model in frame order. The object detection model is an aggregation network based on vision and language. The object detection model only processes the image features corresponding to one frame of traffic accident image in each round of training. The training object detection model calculates the correlation between the image features corresponding to one frame of traffic accident image and the corresponding text features. Specifically, the image features and the text features of the traffic accident image are mapped to a unified feature space, and the two are converted into vectors of the same dimension. Then, the vector distance values ​​corresponding to the two in the unified feature space are calculated. The correlation between the image features and the text features is quantified by the vector distance value. The smaller the vector distance value, the greater the correlation between the corresponding image features and the text features. By calculating and comparing the vector distance values ​​corresponding to the image features and the text features, the text features with the greatest correlation with the image features are obtained. Based on the aggregation algorithm, the text features with the greatest correlation are fused with the image features to form a feature map. By analogy, the feature maps corresponding to all the image features in the traffic accident image can be determined. Based on the feature map corresponding to each image feature, the target detection head algorithm predicts the category and bounding box of each object in the traffic accident image to locate the corresponding area of ​​each object in the traffic accident image. The target text comparison head algorithm obtains one or more feature maps corresponding to the corresponding area of ​​each object, and obtains the similarity corresponding to the corresponding one or more feature maps to determine the correlation corresponding to the object, such as determining the maximum correlation as the correlation corresponding to the object, or taking the mean of the similarity corresponding to the one or more feature maps corresponding to the object as the correlation corresponding to the object. The correlation corresponding to each object in the traffic accident image is comprehensively counted, and the corresponding relationship between the correlation and the correlation confidence is combined to obtain the correlation confidence corresponding to each object. The higher the correlation confidence, the higher the correlation between the image feature and the text feature of the object. Further, a classification loss function is established based on the corresponding predicted category of the object and the corresponding marked category, a bounding box regression loss function is established based on the corresponding predicted bounding box of the object and the corresponding marked bounding box, and the parameters of the object detection model are updated based on the classification loss function and the bounding box regression loss function; at the same time, the correlation confidence corresponding to each object is compared with the correlation confidence threshold, and the parameters of the object detection model are updated based on the comparison result. Return to the step of calculating the correlation between the image features and the several text features respectively, until the classification loss function and the bounding box regression loss function are minimized, that is, the accuracy of the object detection model in identifying and locating the objects matching the traffic accident text set in the traffic accident image reaches an acceptable range, and the correlation confidence reaches the correlation confidence threshold, that is, the image-text correlation of each object reaches the preset correlation target, and the trained object detection model is obtained.

[0035] like Figure 3 As shown, Figure 3 The schematic diagram of traffic accident reasoning based on the trained object detection model provided by the second embodiment of the present invention is shown. The reasoning stage is performed based on the trained object detection model. The trained object detection model receives the traffic accident text and the image to be detected, and detects the text related to the traffic accident text set in the real-time image of the intersection. Since the aggregation network of the object detection model has been trained to make the image features and the text features have a certain correlation, the correlation of the image text is calculated during reasoning, and the area with high correlation is the sensitive area related to the traffic accident text. Specifically, the current frame image and the text related to the traffic accident (such as broken glass, triangle pile, large truck, pedestrian, etc.) are respectively input into the image encoder and the text encoder for processing to obtain the corresponding image features and text features, and the object matching the text in the current frame image is obtained through the visual and language aggregation network of the object detection model, and the category and bounding box of the matched object and the corresponding confidence of the image-text correlation are determined, such as obtaining a heat map associated with the traffic accident vocabulary based on the prompt, and obtaining the confidence of the image-text correlation of the matched object by analyzing the heat map. o If an object matching the text related to the traffic accident is detected in the current frame image, the continuous frames after the current frame image are detected to detect the traffic accident. The trained object detection model is used to obtain the objects matching the text related to the traffic accident in the continuous frames after the current frame image. The confidence of the corresponding image-text correlation is determined and accumulated to obtain the confidence accumulation value. The formula is as follows: conf v = (Formula 1) conf v is the accumulated confidence value. The method of obtaining the objects that match the traffic accident in the consecutive frames after the current frame image is the same as the method of obtaining the objects that match the current frame, and will not be repeated. A confidence threshold for the occurrence of a traffic accident is set in advance. If the accumulated confidence values ​​of sensitive objects in the current frame image and the subsequent frames exceed the confidence threshold for the occurrence of the traffic accident, it is confirmed that a traffic accident has occurred. Otherwise, it is confirmed that no traffic accident has occurred. For example, in a consecutive number of frames, it is detected that a pedestrian lies still for a long time, broken glass appears in several consecutive frames of images, etc., by analyzing the time series of objects matched with text related to the traffic accident, it is determined whether a traffic accident has occurred at the intersection. There is no need to conduct an overall analysis of the real-time image of the intersection, and there is no need to collect a large number of traffic accident images at the intersection. It is also possible to analyze whether a traffic accident has occurred at the intersection, thereby improving the accuracy and efficiency of traffic accident detection.

[0036] A second aspect of the embodiments of the present invention provides a traffic accident detection device. Figure 4 A schematic diagram of a traffic accident detection device provided by an embodiment of the present invention is shown. Figure 4 As shown, it includes: a first module 101, marking each object in a plurality of traffic accident images; a second module 102, calculating the correlation between image features in the traffic accident images and a plurality of traffic accident text features based on an object detection model, and forming a feature map based on the correlation; a third module 103, predicting each object in the traffic accident image based on the feature map, updating the object detection model based on the prediction result and the mark, and returning to the step of calculating the correlation until a preset condition is met; a fourth module 104, performing traffic accident detection through the trained object detection model.

[0037] According to another aspect of the present invention, an electronic device is provided. The electronic device may be a server. Figure 5 As shown. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the traffic accident detection method described above is implemented.

[0038] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium, and its internal structure diagram is shown in FIG. Figure 6 The computer readable storage medium stores a computer program that, when executed by a processor, performs the above traffic accident detection method.

[0039] Finally, it should be noted that a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program of the method for setting system parameters can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program can be a disk, an optical disk, a read-only storage memory (ROM) or a random access memory (RAM), etc. The above-mentioned computer program embodiment can achieve the same or similar effect as any of the above-mentioned method embodiments corresponding thereto.

[0040] In addition, the method disclosed in the embodiment of the present invention can also be implemented as a computer program executed by a processor, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above functions defined in the method disclosed in the embodiment of the present invention are performed.

[0041] In addition, the above method steps and system units may also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to implement the above steps or unit functions.

[0042] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0043] In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes. Computer-readable media include computer storage media and communication media, which include any media that facilitate the transfer of computer programs from one location to another. The storage medium may be any available medium that can be accessed by a general or special-purpose computer. As an example and not by way of limitation, the computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of instructions or data structures and can be accessed by a general or special-purpose computer or a general or special-purpose processor. In addition, any connection may be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves are used to send software from a website, server, or other remote source, the above-mentioned coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are all included in the definition of the medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0044] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0045] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0046] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0047] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0048] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A traffic accident detection method, characterized in that: include: Marking each object in a plurality of traffic accident images; Calculating the correlation between the image features in the traffic accident image and a plurality of traffic accident text features based on the object detection model, and forming a feature map based on the correlation; Predicting each object in the traffic accident image based on the feature map, updating the object detection model based on the prediction result and the label, and returning to the step of calculating the correlation until a preset condition is met; Traffic accident detection using trained object detection models.

2. The traffic accident detection method according to claim 1, characterized in that: The step of calculating the correlation between the image features in the traffic accident image and a plurality of traffic accident text features comprises: Mapping the image features and the plurality of traffic accident text features in the traffic accident image to a unified feature space to convert them into vectors of the same dimension; The vector distance values ​​respectively corresponding to the image features and the plurality of traffic accident text features in the unified feature space are calculated to determine the correlation.

3. The traffic accident detection method according to claim 2, characterized in that: The step of forming a feature map based on the correlation comprises: Determine the minimum vector distance value among the calculated distance values; Determining the text feature with the greatest correlation among the plurality of traffic accident text features based on the minimum vector distance value; The text feature with the greatest correlation is fused with the image feature based on an aggregation algorithm to form a feature map.

4. The traffic accident detection method according to claim 3, characterized in that: The step of predicting each object in the traffic accident image based on the feature map, updating the object detection model based on the prediction result and the mark, and returning to the step of calculating the correlation until a preset condition is met includes: Predicting the category and bounding box of each object in the traffic accident image based on the feature map and locating the corresponding area of ​​each object in the traffic accident image based on the category and bounding box of each object in the traffic accident image; Obtaining the correlation of the feature graph corresponding to the corresponding area, and calculating the correlation confidence corresponding to each object based on the correlation; The object detection model is updated based on the correlation confidence and the label and the step of calculating the correlation is returned until the predicted category and bounding box corresponding to each object approximate the corresponding label value and the correlation confidence reaches a correlation confidence threshold.

5. The traffic accident detection method according to claim 4, characterized in that: The step of performing traffic accident detection using the trained object detection model includes: Based on the current frame image and the text related to the traffic accident, an object matching the text related to the traffic accident in the current frame image is obtained through a trained object detection model, and a category and a bounding box of the matched object and a confidence level of the corresponding image-text correlation are determined; A plurality of continuous frame images after the current frame image are monitored, and traffic accident detection is performed based on the monitoring results of the plurality of continuous frame images.

6. The traffic accident detection method according to claim 5, characterized in that: The step of monitoring a plurality of consecutive frame images after the current frame image and performing traffic accident detection based on the monitoring results of the plurality of consecutive frame images includes: Obtaining objects that match the text related to the traffic accident in a plurality of consecutive frame images after the current frame image through the trained object detection model, respectively determining and accumulating the confidences of the corresponding image-text correlations to obtain a confidence accumulation value; If the accumulated confidence value exceeds the confidence threshold of the traffic accident occurrence, it is confirmed that a traffic accident has occurred.

7. The traffic accident detection method according to claim 1, characterized in that: The method further comprises: Extracting image features from the traffic accident image based on an image encoder, which includes dividing the traffic accident image into a plurality of image blocks and extracting image features from each image block; Several traffic accident text features are extracted from the traffic accident text set based on the text encoder.

8. A traffic accident detection device, characterized in that: The device comprises: The first module is used to mark each object in a plurality of traffic accident images; A second module is used to calculate the correlation between the image features in the traffic accident image and a plurality of traffic accident text features based on the object detection model, and form a feature map based on the correlation; A third module is used to predict each object in the traffic accident image based on the feature map, update the object detection model based on the prediction result and the mark, and return to the step of calculating the correlation until a preset condition is met; The fourth module is used to detect traffic accidents through the trained object detection model.

9. An electronic device, characterized in that: include: at least one processor; and a memory storing computer instructions executable on the processor, wherein the instructions, when executed by the processor, implement the steps of the traffic accident detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the traffic accident detection method according to any one of claims 1 to 7 are performed.

Images