Method and device for determining status information, storage medium and electronic device

By tracking and judging and clustering analysis of image information in traffic accident detection methods, combined with video classification network model, the problem of insufficient robustness of traffic accident detection in the existing technology in complex environments is solved, and the target object status is achieved accurate detection and accident identification.

CN114283361BActive Publication Date: 2024-12-06SHANGHAI SHANMA INTELLIGENT TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111567464.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-20
Publication Date
2024-12-06
Estimated Expiration
2041-12-20

AI Technical Summary

Technical Problem

The existing traffic accident detection methods are not robust enough in complex traffic environments, and cannot accurately determine the target object status in the video image, especially the inability to effectively determine whether the vehicle is in a parking state, resulting in low recognition efficiency and high misidentification rate.

Method used

By obtaining the image information of the target object, tracking and judging and clustering operations, determining the status of the target object, and analyzing the historical frame images using the video classification network model, and obtaining the image sequence of the target area to determine whether there is an accident. The specific steps include: obtaining the image information of the target object, determining whether it is in a parking state, clustering and analyzing the target area, obtaining historical frame images, cutting the image to obtain the target area image sequence, and matching features through the video classification network model to determine the accident.

Benefits of technology

It realizes accurate timely detection of the target object status in complex traffic environments, improves the robustness and accuracy of traffic accident detection, reduces the rate of misidentification, and can quickly and effectively identify the area where traffic accidents occur.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114283361B_ABST
    Figure CN114283361B_ABST
Patent Text Reader

Abstract

The present application discloses a method and device for determining status information, a storage medium and an electronic device, the method comprising: acquiring a certain number of frames of image information, the image information comprising first object information of a target object; based on the first object information, tracking and judging the target object to determine the status of the target object; when it is determined that the target object is in a parking state, clustering the target object through a clustering algorithm based on the second object information of the target object to obtain a target area, and classifying the target area using a video classification network model according to historical data of the target area to obtain target status information; the above technical solution is adopted to solve the problem that the status of the target object in the video image information cannot be accurately determined in the related art.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular, to a method and device for determining status information, a storage medium, and an electronic device. Background Art

[0002] Among the related technologies, traffic accident detection is one of the most important parts of intelligent transportation systems. Real-time and robust traffic accident detection methods can make great contributions to reducing casualties and property losses. With the rapid development of intelligent transportation systems, the research on traffic accident detection systems based on computer vision technology and image processing technology has attracted widespread attention, and many researchers have also made significant progress in this field.

[0003] However, due to the complexity of the traffic environment, the methods currently proposed still have certain limitations in practical application. In the related technology, the existing methods have problems such as insufficient running speed or insufficient robustness in complex traffic environments. Specifically, there are the following defects or deficiencies: using traditional logical judgment methods, such as judging whether an accident has occurred based on whether there are people around the vehicle, is not robust; directly using video classification methods will introduce too much irrelevant background information, making it difficult to focus on the area where the accident occurred.

[0004] With regard to the problem in the related art that the state of the target object in the video image information cannot be accurately determined, no effective solution has been found so far. Summary of the invention

[0005] The embodiments of the present application provide a method and device for determining status information, a storage medium, and an electronic device, so as to at least solve the problem in the related art that the status of a target object in video image information cannot be accurately determined.

[0006] According to an embodiment of the present application, a method for determining status information is provided, including: acquiring image information of a target number of frames, wherein the image information includes first object information of a target object; based on the first object information, tracking and judging the target object to determine the status of the target object; when it is determined that the target object is in a parking state, clustering the target object based on the second object information of the target object to obtain a target area; and classifying the target area according to historical data of the target area to obtain target status information.

[0007] In an exemplary embodiment, the target area is classified and processed based on historical data of the target area to obtain target status information, including: obtaining historical frame images of the target area from the image information, and analyzing the target area image sequence in the historical frame images through a video classification network model to determine whether there is an accident in the target area.

[0008] In an exemplary embodiment, it is determined that the target object is in the parking state by: acquiring first image information of the target object in a previous frame image of the target frame image, and second image information of the target object in the target frame image, wherein the image information includes: the target frame image and the previous frame image; when an intersection-and-union ratio of the first image information and the second image information exceeds a preset threshold, it is determined that the target object is in the parking state.

[0009] In an exemplary embodiment, based on the second object information of the target object, a clustering operation is performed on the target object to obtain a target area, including: obtaining size information and inter-object distance information of the target object, wherein the inter-object distance information is the adjacent distance between the target object and an adjacent object of the target object; and clustering analysis is performed on the size information and inter-object distance information according to a target clustering algorithm to obtain one or more of the target areas.

[0010] In an exemplary embodiment, after obtaining the historical frame images of the target area from the image information, the method further includes: obtaining multiple historical frame images in which the target area exists from the historical frame images of the target area; and obtaining the target area image sequence by cutting out the multiple historical frame images.

[0011] In an exemplary embodiment, the parking area image sequence is obtained by cutting out the multiple historical frame images, comprising: obtaining an analysis criterion of the video classification network model, wherein the analysis criterion is used to indicate the frame number requirement of the video classification network model for an input target area image sequence; and obtaining the target area image sequence by cutting out the multiple historical frame images according to the analysis criterion.

[0012] In an exemplary embodiment, a target area image sequence in the historical frame image is analyzed through a video classification network model to determine whether an accident occurs in the target area, including: inputting the target area image sequence in the historical frame image into the video classification network model; obtaining a first image feature of a pre-saved image sequence in which an accident occurs, and obtaining a second image feature of the target area image sequence through the video classification network model; matching the first image feature with the second image feature through the video classification network model, and determining that an accident occurs in the target area when the similarity between the first image feature and the second image feature is greater than a preset threshold, and determining that no accident occurs in the target area when the similarity between the first image feature and the second image feature is less than a preset threshold.

[0013] According to another embodiment of the embodiments of the present application, a device for determining status information is also provided, including: an acquisition module, used to acquire image information of a target number of frames, wherein the image information includes first object information of the target object; a first determination module, used to track and judge the target object based on the first object information to determine the status of the target object; a second determination module, used to cluster the target object based on the second object information of the target object when it is determined that the target object is in a parking state to obtain a target area; and a processing module, used to classify the target area according to historical data of the target area to obtain target status information.

[0014] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining status information when running.

[0015] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the accident confirmation method through the computer program.

[0016] In an embodiment of the present application, by acquiring a certain number of frames of image information, the image information includes first object information of the target object; based on the first object information, the target object is tracked and judged to determine the state of the target object; when it is determined that the target object is in a parking state, based on the second object information of the target object, a clustering operation is performed on the target object through a clustering algorithm to obtain a target area, and according to the historical data of the target area, a video classification network model is used to classify the target area to obtain target state information; the above technical solution is adopted to solve the problem that the state of the target object in the video image information cannot be accurately determined in the related technology, and it is achieved that the state of the target object can be determined in real time and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of a traffic accident detection system of an optional method for determining status information in an embodiment of the present application;

[0019] Figure 2 is a flowchart of an optional method for determining status information according to an embodiment of the present application;

[0020] Figure 3 is a detection flow chart of an optional method for determining status information according to an embodiment of the present application;

[0021] Figure 4 is a structural diagram of an optional traffic accident detection system in an embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of an execution of determining an optional state information according to an embodiment of the present application;

[0023] Figure 6 It is a structural block diagram of an optional status information determination system according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] The method embodiment provided in the embodiment of the present application can be executed in a traffic accident detection system or a similar computing system. Taking running on a traffic accident detection system as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a traffic accident detection system of a method for determining state information in an embodiment of the present application. Figure 1 As shown, the traffic accident detection system may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing system such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. In an exemplary embodiment, the above-mentioned traffic accident detection system may also include a transmission device 106 for communication functions and an input and output device 108. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above traffic accident detection system. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the state information in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the traffic accident detection system via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission system 106 is used to receive or send data via a network. A specific example of the above network may include a wireless network provided by a communication provider of the traffic accident detection system. In one example, the transmission system 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet.

[0029] In this embodiment, a method for determining state information is provided, which is applied to the above-mentioned traffic accident detection system. Figure 2 : is a flowchart of an optional method for determining status information according to an embodiment of the present application, the process comprising the following steps:

[0030] Step S202, acquiring image information of a target number of frames, wherein the image information includes first object information of a target object;

[0031] Step S204: tracking and judging the target object based on the first object information to determine the state of the target object;

[0032] Step S206, when it is determined that the target object is in a parking state, clustering the target object based on the second object information of the target object to obtain a target area;

[0033] Step S208: classify the target area according to the historical data of the target area to obtain target state information.

[0034] It should be noted that the above-mentioned clustering operation of the target objects can be performed by using a clustering algorithm, and the clustering algorithm may include: DBSCAN (Density-Based Spatial Clustering of Applications with Noise, a representative density-based clustering algorithm), k-means (k-means clustering algorithm, K-means algorithm), GMM (Gaussian Mixture Model, Gaussian mixture model), etc., and this application does not impose any restrictions on this.

[0035] It should be noted that the above-mentioned acquisition of image information of the target number of frames can be used to perform vehicle detection on the image, wherein the detection method can adopt the common deep learning detection network yolo series or Faster-rcnn series, and this application does not impose any restrictions on this.

[0036] It should be noted that the above-mentioned tracking and judgment of the target object can adopt the sort / deep sort multi-target tracking method, and this application does not impose any limitation on this.

[0037] Through the above steps, in the embodiment of the present application, the image information of the target number frames is obtained through vehicle detection, wherein the image information includes the first object information of the target object; based on the first object information, the target object is tracked and judged by the tracking module to determine the state of the target object; when it is determined that the target object is in a parking state, based on the second object information of the target object, the target object is clustered to obtain the target area, and the target area is classified and processed using a video classification network model according to the historical image frame data of the target area to obtain the target state information. The above technical solution solves the problem that the state of the target object in the video image information cannot be accurately determined in the related technology, and achieves the technical effect of being able to accurately determine the state of the target object in real time.

[0038] In an exemplary embodiment, the target area is classified and processed based on historical data of the target area to obtain target status information, including: obtaining historical frame images of the target area from the image information, and analyzing the target area image sequence in the historical frame images through a video classification network model to determine whether there is an accident in the target area.

[0039] That is to say, in an embodiment of the present invention, the traffic accident detection system needs to first obtain historical image frame data of the target area where a traffic accident may occur, and form a target area image sequence consisting of the historical image frames and the current image frames of the target area, and analyze and classify the target area image sequence through a video classification network model to determine whether there is a traffic accident in the target area.

[0040] Based on the above process, it is determined that the target object is in the parking state in the following manner: first image information of the target object in a previous frame image of the target frame image and second image information of the target object in the target frame image are obtained, wherein the image information includes: the target frame image and the previous frame image; when the intersection and union ratio of the first image information and the second image information exceeds a preset threshold, it is determined that the target object is in the parking state.

[0041] When an accident occurs, there are usually onlookers around the accident vehicle, and the accident vehicle will inevitably stop driving. Therefore, by detecting whether there are parked vehicles, it is possible to preliminarily determine whether an accident may have occurred, thereby reducing the workload of the video classification network model; specifically, the specific method for determining whether the target object is in a parked state includes: by obtaining the first image information of the target object in the previous frame image of the target frame image, and the second image information of the target object in the target frame image, when the intersection and union ratio of the first image information and the second image information exceeds a preset threshold, it is determined that the target object is in a parked state.

[0042] In an exemplary embodiment, based on the second object information of the target object, a clustering operation is performed on the target object to obtain a target area, including: obtaining size information and inter-object distance information of the target object, wherein the inter-object distance information is the adjacent distance between the target object and an adjacent object of the target object; and clustering analysis is performed on the size information and inter-object distance information according to a target clustering algorithm to obtain one or more of the target areas.

[0043] After determining that the vehicle to be detected is a parked vehicle, in order to further determine the area where the accident may have occurred, it is necessary to first detect and cluster the parked vehicles around the parked vehicle. In actual applications, there may be onlookers around the traffic accident scene, but the onlookers are generally far away. When clustering parked vehicles, the distance of the parked vehicles will be judged. When the distance is less than the preset value, it will be classified into a parking area to prevent onlookers from being introduced into the recognition image and increasing the recognition workload. Figure 5It can be seen that by detecting all parked vehicles around the parked vehicle and judging that the distance of the parked vehicle is less than a predetermined threshold, the parked vehicle is divided into the parking area. Since there may be multiple parking gathering areas at the scene of the accident, all existing parking areas are obtained for further detection.

[0044] In an exemplary embodiment, after obtaining the historical frame images of the target area from the image information, the method further includes: obtaining multiple historical frame images in which the target area exists from the historical frame images of the target area; and obtaining the target area image sequence by cutting out the multiple historical frame images.

[0045] It is understandable that accurate detection results cannot be obtained by only detecting the image of the parking area at the target moment. Therefore, the traffic accident detection system first obtains multiple historical frame images of the parking area from the cached historical frame images, and cuts out the parking area image from the multiple historical frame images to facilitate subsequent further identification and classification.

[0046] Based on the above process, the parking area image sequence is obtained by cutting out the multiple historical frame images, including: obtaining an analysis standard of the video classification network model, wherein the analysis standard is used to indicate the frame number requirement of the video classification network model for the input target area image sequence; and obtaining the target area image sequence by cutting out the multiple historical frame images according to the analysis standard.

[0047] Since different video classification networks have different requirements for input images, in order to help the video classification network quickly complete the analysis, before cutting out the parking area image sequence from multiple historical frame images, the analysis standard of the video classification network is first obtained, specifically including the frame number requirement for the input parking area image sequence, and the parking area image sequence is obtained according to the analysis standard.

[0048] In an exemplary embodiment, a target area image sequence in the historical frame image is analyzed through a video classification network model to determine whether an accident occurs in the target area, including: inputting the target area image sequence in the historical frame image into the video classification network model; obtaining a first image feature of a pre-saved image sequence in which an accident occurs, and obtaining a second image feature of the target area image sequence through the video classification network model; matching the first image feature with the second image feature through the video classification network model, and determining that an accident occurs in the target area when the similarity between the first image feature and the second image feature is greater than a preset threshold, and determining that no accident occurs in the target area when the similarity between the first image feature and the second image feature is less than a preset threshold.

[0049] After acquiring the target area image sequence of the target parking area, the target area image sequence is input into the video classification network model. By acquiring the first image feature of the image sequence where the accident currently occurs and the second image feature of the target area image sequence, the first image feature and the second image feature are matched. When the similarity between the first image feature and the second image feature is less than a preset threshold, it is determined that no accident has occurred in the target area.

[0050] It should be noted that after determining that an accident has occurred in the target area, the traffic accident detection system will also archive this information to facilitate subsequent information search, and the archived information may include: parking area image sequence, video classification results, target time, and the location of the accident, etc. This application does not limit this.

[0051] In an exemplary embodiment, obtaining the parking area image sequence by cutting out the multiple historical frame images includes: acquiring an analysis criterion of the video classification network, wherein the analysis criterion is used to indicate the frame number requirement of the video classification network for the input parking area image sequence; and obtaining the parking area image sequence by cutting out the multiple historical frame images according to the analysis criterion.

[0052] Since different video classification networks have different requirements for input images, in order to help the video classification network quickly complete the analysis, before cutting out the parking area image sequence from multiple historical frame images, the analysis standard of the video classification network is first obtained, specifically including the frame number requirement for the input parking area image sequence, and the parking area image sequence is obtained according to the analysis standard.

[0053] It should be noted that the above-mentioned video classification network model belongs to a deep learning video classification network, which can be a 2D network, such as a TSN model; a 3D network, such as an I3D model, and this application does not impose any restrictions on this.

[0054] It should be noted that the above-mentioned parking area image sequence is obtained by extracting the images of the parking area one by one from the historical image sequence. The parking area image sequence can be 8 frames or 16 frames, depending on the deep learning video classification network used, and this application does not impose any restrictions on this.

[0055] Based on the above process, after analyzing the parking area image sequence in the historical frame image through the video classification network model to determine whether there is an accident in the parking area, the method also includes: when it is determined that an accident has occurred in the parking area, performing image recognition on the target frame image to obtain description information of the accident, wherein the description information includes at least one of the following: the place where the accident occurred, the license plate information of the vehicle where the accident occurred; initiating an alarm operation based on the description information.

[0056] After determining that a traffic accident has occurred, the traffic accident detection system will continue to perform image recognition on the target frame image to obtain information such as the address information of the accident site, the license plate information of the accident site, etc., and use this information to initiate an alarm operation to remind people to take countermeasures.

[0057] It should be noted that after obtaining this information and issuing an alarm, the traffic accident detection system will also archive this information to facilitate subsequent information search. The archived information may include: parking area image sequence, video classification results, target time, and the location of the accident. This application does not limit this.

[0058] In order to better understand the technical solutions of the above embodiments, the following terms are explained in the embodiments of the present invention.

[0059] Robustness: The ability of a system to survive abnormal and hazardous situations.

[0060] Robustness: refers to the ability of a control system to maintain certain other performance characteristics under certain (structure, size) parameter perturbations.

[0061] In order to better understand the process of the method for determining the above-mentioned status information, the implementation method flow of determining the above-mentioned status information is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0062] In this embodiment, a method for determining status information is provided. Figure 3 is a detection flow chart of a method for determining status information according to an embodiment of the present application, such as Figure 3 As shown, the specific steps are as follows:

[0063] Step S302: Start accident detection;

[0064] Step S304: inputting the video stream to the traffic accident detection unit;

[0065] Step S306: performing vehicle detection and tracking on the image;

[0066] Step S308: using the tracking information to find the parked vehicle;

[0067] Step S310: Determine whether there is a parked vehicle, if not, continue to return to step S304;

[0068] Step S312: If there are parked vehicles, cluster the parked vehicles into parking areas using a clustering algorithm;

[0069] Step S314: extracting a parking area ROI (Region of interest) image sequence from the cached historical video frames;

[0070] Step S316: using a video classification network to classify the image sequence to determine whether an accident has occurred, and if no accident has occurred, returning to step S304;

[0071] Step S318: If it is determined that an accident has occurred, an alarm is issued through the alarm unit;

[0072] Step S320: Buffering the video stream while inputting the video stream for detection.

[0073] In an embodiment of the present invention, the traffic accident detection system continuously acquires video streams through a video stream acquisition unit and inputs the video streams into the traffic accident detection unit. The traffic accident detection unit determines whether there is a parked vehicle by performing vehicle detection and tracking on the images in the video stream. If not, the detection of the next video stream continues. If there is a parked vehicle, the parked vehicles are clustered into parking areas to obtain parking areas, and an image sequence of parking areas is captured from the cached historical video frames. The image sequence is classified using a video classification network to determine whether a traffic accident has occurred. If no traffic accident has occurred, the detection of the next video stream continues. If it is determined that an accident has occurred, an alarm is issued through the alarm unit.

[0074] Through the above steps, the state of the target object can be determined in real time and accurately, which solves the problem in the related art that the state of the target object in the video image information cannot be accurately determined.

[0075] Figure 4 FIG. 4 is a structural diagram of an optional traffic accident detection system in an embodiment of the present invention; Figure 4 As shown, specifically including:

[0076] Video stream acquisition unit: used to continuously acquire traffic video stream and input the video stream into the traffic accident detection unit;

[0077] Traffic accident detection unit: receives the video stream transmitted by the video stream acquisition unit, and detects each frame image in the video stream; when it is determined that there is a parked vehicle in the target frame image in the video stream, clusters the parked vehicles in the target frame image using a clustering algorithm to obtain a parking area, wherein the parking area includes: the parked vehicle, and adjacent vehicles adjacent to the parked vehicle, and the target frame image corresponds to a target moment; obtains the historical frame images of the parking area before the target moment from the video stream, and analyzes the parking area image sequence in the historical frame images through a video classification network model to determine whether there is an accident in the parking area.

[0078] Alarm unit: when it is determined that an accident occurs in the parking area, an alarm is issued through the alarm unit; and corresponding instructions can be executed according to the accident level output by the traffic accident detection unit.

[0079] Event retrieval and query unit: when it is determined that an accident has occurred in the parking area, the parking area image sequence, the video classification result, the target time and the location of the accident are integrated and archived; and the parking area image sequence and the video classification result are input into the video classification network model, and the video classification network model is further strengthened through deep learning.

[0080] Figure 5 is a schematic diagram of an optional execution of determining status information according to an embodiment of the present application; Figure 5 As shown, specifically including:

[0081] When a traffic accident occurs, one or more cars will stop on the road to deal with the accident or cannot drive away due to the accident. At this time, the detection algorithm detects the parked vehicles on the road, and clusters all the detected parked vehicles into one or more cluster areas through the clustering algorithm, such as Figure 5 As shown in the figure, two cars collided on the highway and stopped on the right side of the road to deal with the accident. The traffic accident detection system detected and identified the two vehicles through the input video stream, divided the parking area, and then classified the video of the parking area to obtain the accident detection results and output them.

[0082] In an embodiment of the present invention, a system for detecting whether an accident has occurred is proposed, namely, a traffic accident detection system. The traffic accident detection system determines the parking area by detecting parked vehicles and obtains historical frame images of the parking area. The image sequence of the parking area is input into a video classification network model for identification and classification to determine whether an accident has occurred. By accurately locating the potential accident area, the computational pressure of the subsequent video classification network is reduced. In addition, the video classification network is used to determine whether an accident has occurred. Compared with the logical judgment method, the robustness is better.

[0083] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0084] Figure 6 is a structural block diagram of a device for determining status information according to an embodiment of the present application; Figure 6 As shown, including:

[0085] An acquisition module 62 is used to acquire image information of a target number of frames, wherein the image information includes first object information of a target object;

[0086] A first determination module 64 is used to track and determine the target object based on the first object information to determine the state of the target object;

[0087] A second determination module 66 is used to perform a clustering operation on the target object based on the second object information of the target object to obtain a target area when it is determined that the target object is in a parking state;

[0088] The processing module 68 is used to classify the target area according to the historical data of the target area to obtain target state information.

[0089] Through the above-mentioned device, by acquiring a certain number of frames of image information, the image information includes first object information of the target object; based on the first object information, the target object is tracked and judged to determine the state of the target object; when it is determined that the target object is in a parking state, based on the second object information of the target object, a clustering operation is performed on the target object through a clustering algorithm to obtain a target area, and according to the historical data of the target area, the target area is classified and processed using a video classification network model to obtain target state information; the above-mentioned technical solution is adopted to solve the problem that the state of the target object in the video image information cannot be accurately determined in the related technology, thereby realizing the ability to determine the state of the target object in real time and accurately.

[0090] In an exemplary embodiment, the processing module is also used to classify the target area to obtain target status information, including: obtaining historical frame images of the target area from the image information, and analyzing the target area image sequence in the historical frame images through a video classification network model to determine whether there is an accident in the target area.

[0091] That is to say, in an embodiment of the present invention, the traffic accident detection system needs to first obtain historical image frame data of the target area where a traffic accident may occur, and form a target area image sequence consisting of the historical image frames and the current image frames of the target area, and analyze and classify the target area image sequence through a video classification network model to determine whether there is a traffic accident in the target area.

[0092] In an exemplary embodiment, the first determination module is further used to determine that the target object is in the parking state by: acquiring first image information of the target object in a previous frame image of the target frame image, and second image information of the target object in the target frame image, wherein the image information includes: the target frame image and the previous frame image; when the intersection and union ratio of the first image information and the second image information exceeds a preset threshold, determining that the target object is in the parking state.

[0093] When an accident occurs, there are usually onlookers around the accident vehicle, and the accident vehicle will inevitably stop driving. Therefore, by detecting whether there are parked vehicles, it is possible to preliminarily determine whether an accident may have occurred, thereby reducing the workload of the video classification network model; specifically, the specific method for determining whether the target object is in a parked state includes: by obtaining the first image information of the target object in the previous frame image of the target frame image, and the second image information of the target object in the target frame image, when the intersection and union ratio of the first image information and the second image information exceeds a preset threshold, it is determined that the target object is in a parked state.

[0094] In an exemplary embodiment, the second determination module is further used to perform a clustering operation on the target object based on the second object information of the target object to obtain a target area, including: obtaining size information and inter-object distance information of the target object, wherein the inter-object distance information is the adjacent distance between the target object and an adjacent object of the target object; and performing cluster analysis on the size information and inter-object distance information according to a target clustering algorithm to obtain one or more of the target areas.

[0095] After determining that the vehicle to be detected is a parked vehicle, in order to further determine the area where the accident may have occurred, it is necessary to first detect and cluster the parked vehicles around the parked vehicle. In actual applications, there may be onlookers around the traffic accident scene, but the onlookers are generally far away. When clustering parked vehicles, the distance of the parked vehicles will be judged. When the distance is less than the preset value, it will be classified into a parking area to prevent onlookers from being introduced into the recognition image and increasing the recognition workload. Figure 5 It can be seen that by detecting all parked vehicles around the parked vehicle and judging that the distance of the parked vehicle is less than a predetermined threshold, the parked vehicle is divided into the parking area. Since there may be multiple parking gathering areas at the scene of the accident, all existing parking areas are obtained for further detection.

[0096] In an exemplary embodiment, the acquisition module is also used to acquire the historical frame images of the target area from the image information, and the method further includes: acquiring multiple historical frame images in which the target area exists from the historical frame images of the target area; and obtaining the target area image sequence by cutting out the multiple historical frame images.

[0097] It is understandable that accurate detection results cannot be obtained by only detecting the image of the parking area at the target moment. Therefore, the traffic accident detection system first obtains multiple historical frame images of the parking area from the cached historical frame images, and cuts out the parking area image from the multiple historical frame images to facilitate subsequent further identification and classification.

[0098] In an exemplary embodiment, the acquisition module is also used to obtain the parking area image sequence by cutting out the multiple historical frame images, including: obtaining the analysis standard of the video classification network model, wherein the analysis standard is used to indicate the frame number requirement of the video classification network model for the input target area image sequence; and obtaining the target area image sequence by cutting out the multiple historical frame images according to the analysis standard.

[0099] Since different video classification networks have different requirements for input images, in order to help the video classification network quickly complete the analysis, before cutting out the parking area image sequence from multiple historical frame images, the analysis standard of the video classification network is first obtained, specifically including the frame number requirement for the input parking area image sequence, and the parking area image sequence is obtained according to the analysis standard.

[0100] In an exemplary embodiment, the processing module is also used to analyze the target area image sequence in the historical frame image through a video classification network model to determine whether there is an accident in the target area, including: inputting the target area image sequence in the historical frame image into the video classification network model; obtaining a first image feature of a pre-saved image sequence in which an accident occurred, and obtaining a second image feature of the target area image sequence through the video classification network model; matching the first image feature and the second image feature through the video classification network model, and determining that an accident has occurred in the target area when the similarity between the first image feature and the second image feature is greater than a preset threshold, and determining that no accident has occurred in the target area when the similarity between the first image feature and the second image feature is less than a preset threshold.

[0101] After acquiring the target area image sequence of the target parking area, the target area image sequence is input into the video classification network model. By acquiring the first image feature of the image sequence where the accident currently occurs and the second image feature of the target area image sequence, the first image feature and the second image feature are matched. When the similarity between the first image feature and the second image feature is less than a preset threshold, it is determined that no accident has occurred in the target area.

[0102] In an exemplary embodiment, the acquisition module is also used to obtain the parking area image sequence by cutting out the multiple historical frame images, including: obtaining an analysis standard of the video classification network, wherein the analysis standard is used to indicate the frame number requirement of the video classification network for the input parking area image sequence; and obtaining the parking area image sequence by cutting out the multiple historical frame images according to the analysis standard.

[0103] Since different video classification networks have different requirements for input images, in order to help the video classification network quickly complete the analysis, before cutting out the parking area image sequence from multiple historical frame images, the analysis standard of the video classification network is first obtained, specifically including the frame number requirement for the input parking area image sequence, and the parking area image sequence is obtained according to the analysis standard.

[0104] In an exemplary embodiment, the acquisition module is also used to analyze the parking area image sequence in the historical frame image through a video classification network model to determine whether there is an accident in the parking area. The method also includes: when it is determined that an accident has occurred in the parking area, performing image recognition on the target frame image to obtain description information of the accident, wherein the description information includes at least one of the following: the place where the accident occurred, the license plate information of the vehicle where the accident occurred; and initiating an alarm operation based on the description information.

[0105] After determining that a traffic accident has occurred, the traffic accident detection system will continue to perform image recognition on the target frame image to obtain information such as the address information of the accident site, the license plate information of the accident site, etc., and use this information to initiate an alarm operation to remind people to take countermeasures.

[0106] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0107] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0108] S1, acquiring image information of a target number of frames, wherein the image information includes first object information of a target object;

[0109] S2, based on the first object information, tracking and judging the target object to determine the state of the target object;

[0110] S3, when it is determined that the target object is in a parking state, performing a clustering operation on the target object based on the second object information of the target object to obtain a target area;

[0111] S4, classifying the target area according to the historical data of the target area to obtain target state information.

[0112] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0113] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0114] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0115] S1, acquiring image information of a target number of frames, wherein the image information includes first object information of a target object;

[0116] S2, based on the first object information, tracking and judging the target object to determine the state of the target object;

[0117] S3, when it is determined that the target object is in a parking state, performing a clustering operation on the target object based on the second object information of the target object to obtain a target area;

[0118] S4, classifying the target area according to the historical data of the target area to obtain target state information.

[0119] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0120] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0121] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented with a general computing system, they can be concentrated on a single computing system, or distributed on a network composed of multiple computing systems, and optionally, they can be implemented with program codes executable by the computing system, so that they can be stored in a storage system and executed by the computing system, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0122] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining status information, characterized in that: include: Acquire image information of a target number of frames, wherein the image information includes first object information of a target object; Based on the first object information, tracking and judging the target object to determine the state of the target object; When it is determined that the target object is in a parking state, clustering the target object based on the second object information of the target object to obtain a target area; Classify the target area according to the historical data of the target area to obtain target state information; According to the historical data of the target area, the target area is classified to obtain the target state information, including: Acquire a historical frame image of the target area from the image information, and analyze a target area image sequence in the historical frame image through a video classification network model to determine whether there is an accident in the target area; The process of analyzing the target area image sequence in the historical frame image by using a video classification network model to determine whether there is an accident in the target area includes: Inputting the target area image sequence in the historical frame image into the video classification network model; Acquire a first image feature of a pre-saved image sequence of an accident through the video classification network model, and acquire a second image feature of the target area image sequence; The first image feature and the second image feature are matched through the video classification network model. When the similarity between the first image feature and the second image feature is greater than a preset threshold, it is determined that an accident has occurred in the target area. When the similarity between the first image feature and the second image feature is less than the preset threshold, it is determined that no accident has occurred in the target area.

2. The method for determining status information according to claim 1, characterized in that: Determine that the target object is in the parking state by: Acquire first image information of a target object in a previous frame image of a target frame image, and second image information of the target object in the target frame image, wherein the image information includes: the target frame image and the previous frame image; When an intersection-over-union ratio of the first image information and the second image information exceeds a preset threshold, it is determined that the target object is in a parking state.

3. The method for determining status information according to claim 1, characterized in that: Based on the second object information of the target object, clustering the target object to obtain a target area includes: Acquire size information and inter-object distance information of the target object, wherein the inter-object distance information is the adjacent distance between the target object and an adjacent object of the target object; The size information and the distance information between objects are clustered and analyzed according to a target clustering algorithm to obtain one or more target areas.

4. The method for determining status information according to claim 1, characterized in that: After acquiring the historical frame image of the target area from the image information, the method further includes: Acquire a plurality of historical frame images in which the target area exists from the historical frame images of the target area; The target area image sequence is obtained by cutting out images from the multiple historical frame images.

5. The method for determining status information according to claim 4, characterized in that: The step of obtaining the target area image sequence by clipping the multiple historical frame images comprises: Acquire an analysis standard of the video classification network model, wherein the analysis standard is used to indicate the frame number requirement of the video classification network model for an input target area image sequence; The target area image sequence is obtained by clipping the multiple historical frame images according to the analysis standard.

6. A device for determining status information, characterized in that: include: An acquisition module, used for acquiring image information of a target number of frames, wherein the image information includes first object information of a target object; A first determination module, configured to track and determine the target object based on the first object information to determine the state of the target object; A second determination module is used to perform a clustering operation on the target object based on the second object information of the target object to obtain a target area when it is determined that the target object is in a parking state; A processing module, used for classifying the target area according to the historical data of the target area to obtain target state information; Wherein, the processing module is further used to obtain the historical frame images of the target area from the image information, and analyze the target area image sequence in the historical frame images through the video classification network model to determine whether there is an accident in the target area; Among them, the processing module is also used to input the target area image sequence in the historical frame image into the video classification network model; obtain the first image feature of the pre-saved image sequence where the accident occurred, and obtain the second image feature of the target area image sequence through the video classification network model; match the first image feature and the second image feature through the video classification network model, and when the similarity between the first image feature and the second image feature is greater than a preset threshold, determine that an accident has occurred in the target area; when the similarity between the first image feature and the second image feature is less than a preset threshold, determine that no accident has occurred in the target area.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 5 when executed.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 5 through the computer program.