Data detection method, device, storage medium and electronic device

Abstract

Embodiments of the present invention provide a data detection method, device, storage medium and electronic device, which obtain target area data, detect the target area data through a detection model, obtain first feature information of the target area, wherein the first feature information includes depth features of all target objects in the target area, perform multi-target association on the first feature information to establish a target association group, and detect the target association group based on a feature database to obtain a detection result, wherein the feature database includes feature information of the target object, thereby improving the accuracy of data detection.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing, and in particular, to a data detection method, device, storage medium and electronic device. Background Art

[0002] Motor vehicles and non-motor vehicles are both commonly used transportation tools. Traffic rules for motor vehicles have been developed to a relatively mature level, while traffic management for non-motor vehicles is still in its early stages of development. As the number of non-motor vehicles has increased dramatically, the management data on non-motor vehicle traffic violations has also grown exponentially, thus putting forward new requirements for data management.

[0003] For example, for new types of traffic violations such as overloading of non-motor vehicles, the current main way to identify violations is through manual monitoring. That is, law enforcement officers conduct supervision at intersections where cases often occur and detect abnormal violations through the human eye. Manual monitoring is the current mainstream way to identify non-motor vehicle violations, which requires a lot of manpower and material resources and has a high underreporting rate.

[0004] However, if neural network deep learning is used for supervision, it will be limited by the complexity of actual application scenarios, with too many false alarms and poor practicality.

[0005] Therefore, how to effectively improve the validity of data is one of the main problems that need to be solved in this field. Summary of the invention

[0006] According to an embodiment of the present invention, a data detection method is provided, which solves the problem of low data validity in current data management by performing multi-target association on the first feature information to establish a target association group.

[0007] According to an embodiment of the present invention, a data detection method is provided, which performs re-detection on the data through classification detection to solve the problem of misjudgment of the target object in the target association group.

[0008] According to an embodiment of the present invention, a data detection method is provided, wherein the feature database can be continuously updated according to the detection results, and the detection range can be continuously expanded.

[0009] According to an embodiment of the present invention, a data detection method is provided, which identifies the overloading behavior of non-motor vehicles through data detection, replaces manual detection, reduces manpower and material costs, and improves detection efficiency.

[0010] According to one embodiment of the present invention, there is provided a data detection method, comprising: acquiring target area data, detecting the target area data through a detection model to obtain first feature information of the target area, wherein the first feature information includes depth features of all target objects in the target area; performing multi-target association on the first feature information to establish a target association group; and detecting the target association group based on a feature database to obtain a detection result, wherein the feature database includes feature information of the target object.

[0011] According to an exemplary embodiment of the present invention, multi-target association is performed on the first feature information to establish a target association group, including: performing multi-target tracking on the first feature information to obtain second feature information, wherein the second feature information includes an identifier and a motion trajectory of the target object; and performing association processing on the second feature information to obtain the target association group.

[0012] According to an exemplary embodiment of the present invention, performing association processing on the second characteristic information to obtain the target association group includes: setting a target association area based on the first characteristic information and the second characteristic information;

[0013] Based on the target association area, performing association screening on the target object to obtain the target association group;

[0014] The target association group is updated to obtain a new target association group.

[0015] According to an exemplary embodiment of the present invention, the target association group is detected based on a feature database to obtain a detection result, wherein the feature database includes feature information of the target object, including:

[0016] Based on the feature database, matching the types of the target objects in the target association group to obtain matching results, wherein the matching results include a first matching result and a second matching result;

[0017] Based on the matching result, the first matching result is detected using a first reference threshold, and the second matching result is detected using a second reference threshold to obtain a first detection result;

[0018] Based on the first detection result, the target association group is classified and detected to obtain a second detection result.

[0019] According to an exemplary embodiment of the present invention, it also includes:

[0020] Based on the detection result, the feature database is updated.

[0021] According to another embodiment of the present invention, there is provided a data detection device, comprising:

[0022] A first acquisition module is used to acquire target area data, detect the target area data through a detection model, and obtain first feature information of the target area, wherein the first feature information includes depth features of all target objects in the target area;

[0023] A target association module, used for performing multi-target association on the first feature information to establish a target association group;

[0024] The detection module detects the target association group based on a feature database to obtain a detection result, wherein the feature database includes feature information of the target object.

[0025] According to an exemplary embodiment of the present invention, the target association module includes:

[0026] A multi-target tracking module, configured to perform multi-target tracking on the first feature information to obtain second feature information, wherein the second feature information includes an identifier and a motion trajectory of the target object;

[0027] An association processing module is used to perform association processing on the second feature information to obtain the target association group.

[0028] According to an exemplary embodiment of the present invention, the detection module includes:

[0029] A matching unit, configured to match the type of the target object in the target association group based on the feature database to obtain a matching result, wherein the matching result includes a first matching result and a second matching result;

[0030] A first detection unit, configured to detect the first matching result based on the matching result using a first reference threshold, and detect the second matching result using a second reference threshold, to obtain a first detection result;

[0031] A classification detection unit is used to perform classification detection on the target association group based on the first detection result to obtain a second detection result.

[0032] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0033] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a hardware structure block diagram of a mobile terminal according to a data detection method of an embodiment of the present invention;

[0035] Figure 2 is a flow chart of a data detection method according to an embodiment of the present invention;

[0036] Figure 3 is a flow chart of a data detection method according to an embodiment of the present invention;

[0037] Figure 4 is a flow chart of a data detection method according to an embodiment of the present invention;

[0038] Figure 5 is a flow chart of a data detection method according to an embodiment of the present invention;

[0039] Figure 6 is a structural block diagram of a data detection device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0041] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a detection method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0042] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a detection method in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0043] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0044] From the above background technology, it can be seen that how to effectively detect the behavior of non-motor vehicle passengers from traffic data, reduce misjudgment behavior during the detection process, and reduce the manpower and material costs during the detection process are the main problems that need to be solved at present.

[0045] In order to better solve the above technical problems, the present invention discloses a data detection method, device, storage medium and electronic device, which will be described in detail one by one in the following embodiments.

[0046] According to one embodiment of the present invention, the main application scenario of the data detection method is to detect whether a non-motor vehicle has any violation behavior based on the detection results. The violation behavior mainly refers to the non-motor vehicle exceeding the prescribed number of passengers, that is, illegal overloading behavior.

[0047] See also Figure 2 , Figure 2 A flow chart of a data detection method provided according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0048] S202, acquiring target area data, detecting the target area data through a detection model, and obtaining first feature information of the target area, wherein the first feature information includes depth features of all target objects in the target area;

[0049] According to an embodiment of the present invention, the target area data refers to image data within the target area, wherein the image data includes but is not limited to picture data and video data, and the target area data is obtained by receiving and decoding data information transmitted by a data acquisition device. The data acquisition device may refer to a data acquisition device such as a camera or a video camera. In this embodiment, the image data is mainly obtained by decoding data information collected by the camera or the video camera. The target area can be set according to specific needs by setting the installation position of the data acquisition device.

[0050] According to an embodiment of the present invention, the detection model is a deep learning detection algorithm model, such as the YOLOX detection algorithm, which is pre-trained with the ImageNet dataset, and then fine-tuned with non-motor vehicles and human data in traffic scenes, and finally the detector outputs information such as coordinate frames, positions, and confidences of two categories of non-motor vehicles and human bodies. By detecting the target area data with the detection model, all object points of non-motor vehicles and humans in the target area data can be detected, and the feature information of non-motor vehicles and humans corresponding to the object points can be detected and output.

[0051] According to one embodiment of the present invention, the non-motor vehicles and humans are the target objects that need to be detected in the target area, wherein the first feature information refers to the depth features of all the target objects in the target area data, namely, non-motor vehicles and humans, and the depth features include but are not limited to features such as type, size, position, timestamp and confidence of non-motor vehicles, and features such as size, position, timestamp and confidence of humans.

[0052] S204, performing multi-target association on the first feature information to establish a target association group;

[0053] refer to Figure 3 As shown, step S204 includes:

[0054] S2042, performing multi-target tracking on the first feature information to obtain second feature information, wherein the second feature information includes an identifier and a motion trajectory of the target object;

[0055] According to an embodiment of the present invention, the second feature information of the target object is obtained by performing multi-target tracking on the first feature information. That is, the second feature information of the target object is obtained by multi-target tracking processing based on the depth features of the target objects, i.e., non-motor vehicles and people, i.e., the type, size, position, timestamp, and confidence of the non-motor vehicles, and the size, position, timestamp, and confidence of people, wherein the second feature information includes the IDs and motion trajectories of all the target objects in the target area.

[0056] According to one embodiment of the present invention, the first feature information is tracked by using the IouTrack tracker to assign a unique identifier, i.e., the ID of the target object, to the target object and form a motion track of the target object represented by each ID. In other embodiments provided by the present invention, other multi-target tracking algorithms can also be used to perform multi-target tracking on the first feature information. As long as the same function can be achieved, the present invention is applicable and is not limited here.

[0057] S2044: Perform association processing on the second characteristic information to obtain the target association group.

[0058] According to an embodiment of the present invention, due to the large occlusion of the human body on the non-motor vehicle and the influence of the installation positions of different data acquisition devices, continuous detection is difficult and the continuity of the tracked ID is difficult to ensure, that is, it is impossible to form an effective and continuous motion trajectory. Therefore, in this embodiment, it is necessary to perform association processing on the second feature information after performing multi-target tracking processing on the first feature information, and improve the accuracy of detection by obtaining a target association group.

[0059] According to an embodiment of the present invention, referring to Figure 3 As shown, step S2044 specifically includes:

[0060] a) setting a target association area based on the first feature information and the second feature information; b) performing association screening on the target object based on the target association area to obtain the target association group;

[0061] c) Updating the target association group to obtain a new target association group.

[0062] According to one embodiment of the present invention, step a) can be specifically implemented as follows: according to the second feature information, the ID of the non-motor vehicle is set as the main ID, and an association threshold is set, which may refer to a distance threshold. The target association area is formed according to the association threshold with the main ID as the center.

[0063] According to one embodiment of the present invention, step b) can be specifically implemented by setting association conditions with the main ID as the center, screening the target objects in the target association area according to the association conditions, and setting the target objects that meet the association conditions as the sub-ID corresponding to the main ID.

[0064] According to one embodiment of the present invention, the association conditions include time conditions, trajectory similarity conditions, direction conditions and speed conditions. When the target object in the target association area meets all of the above-mentioned association conditions, the target object is set to the sub-ID corresponding to the main ID.

[0065] According to an embodiment of the present invention, the association condition may include a trajectory similarity condition. The similarity between the human body trajectory and the non-motor vehicle trajectory is calculated through a trajectory similarity matching algorithm model, and a trajectory similarity threshold is set. For example, the trajectory similarity threshold is set to 90%. Based on the first feature information and the second feature information, that is, the deep feature of the target object and the ID and corresponding trajectory of the target object, the trajectory similarity is calculated. When the trajectory similarity is greater than (or greater than or equal to) 90%, the target object meets the trajectory similarity condition. When the target object does not meet the trajectory similarity condition, the association screening is no longer performed. The trajectory similarity matching algorithm model may be a Fréchet distance algorithm, or other algorithms with similar functions, which are not limited here. It is worth mentioning that the trajectory similarity threshold can be set according to specific circumstances and needs.

[0066] According to an embodiment of the present invention, the association condition may include a time condition, and the time condition may be specifically set as a time threshold, for example, the time threshold is set to 10s, and the duration of similar trajectories of the human body and non-motor vehicle is calculated based on the first feature information and the second feature information, i.e., the depth feature of the target object and the ID and corresponding trajectory of the target object. If the duration is greater than (or greater than or equal to) 10s, the target object satisfies the time condition. If the duration is less than 10s, it is judged that the time condition is not met, and the association screening is no longer performed. It is worth mentioning that the time threshold can be set specifically according to specific circumstances and needs.

[0067] According to an embodiment of the present invention, the association condition may include a direction condition, and the direction condition may be specifically set to a motion direction deviation threshold, for example, the motion direction deviation threshold is set to 5%, and based on the first feature information and the second feature information, i.e., the depth feature of the target object and the ID and corresponding trajectory of the target object, the motion direction deviation of similar trajectories of the human body and non-motor vehicle is calculated. If the motion direction deviation is less than (or less than or equal to) 5%, the target object satisfies the direction condition. If the motion direction deviation is greater than 5%, it is determined that the direction condition is not met, and the association screening is no longer performed. It is worth mentioning that the motion direction deviation threshold can be set specifically according to specific circumstances and needs.

[0068] According to an embodiment of the present invention, the association condition may include a speed condition, and the speed condition may be specifically set to a motion speed deviation threshold, for example, the motion speed deviation threshold is set to 5%, and based on the first feature information and the second feature information, i.e., the depth feature of the target object and the ID and corresponding trajectory of the target object, the motion speed deviation of similar trajectories of the human body and non-motor vehicle is calculated. If the motion speed deviation is less than (or less than or equal to) 5%, the target object satisfies the speed condition. If the motion speed deviation is greater than 5%, it is determined that the speed condition is not met, and the association screening is no longer performed. It is worth mentioning that the motion speed deviation threshold can be specifically set according to specific circumstances and needs.

[0069] In some other embodiments of the present invention, the association conditions may include other conditions besides the time condition, the trajectory similarity condition, the direction condition and the speed condition, that is, the conditions may be increased or decreased, and the association conditions may be set according to specific requirements. The above-listed association conditions are not intended to be limiting.

[0070] In some other embodiments of the present invention, the association condition may also include only some of the time condition, the trajectory similarity condition, the direction condition and the speed condition, that is, the conditions may be screened out. The association condition may be set according to specific requirements. The above-listed association conditions are not intended to be limiting.

[0071] According to one embodiment of the present invention, with the main ID as the center, when the target object in the target association area meets the association condition set when performing association screening, the target object of the association condition is set to the sub-ID corresponding to the main ID, and the main ID and the sub-ID constitute the target association group.

[0072] According to one embodiment of the present invention, the step c) can be specifically implemented as performing continuous association screening on the target area. For different sub-IDs that meet the association conditions after association screening, if it is a new sub-ID, the depth features of the human body represented by the sub-ID are extracted and matched with the human body features of the target association group. If the match is successful, the trajectory of the new sub-ID is updated to the corresponding sub-ID in the target association group. If the match fails, the sub-ID is included in the target association group as a new sub-ID. That is, if it is an old sub-ID, the motion trajectory of the corresponding sub-ID in the target association group is updated. The above implementation process updates the target association group to obtain a new target association group.

[0073] It is worth mentioning that the sub-ID of the target association group may disappear briefly in the target area due to reasons such as being blocked. When the blockage disappears, the sub-ID reappears. Therefore, it is necessary to avoid adding the same sub-ID to the target association group. Otherwise, two identical sub-IDs will appear in the target association group, which will have a significant impact on the subsequent non-motor vehicle overload detection results and greatly reduce the accuracy of the detection results. This situation can be effectively detected through step c). When the sub-ID appears again, the trajectory of the new sub-ID is updated to the corresponding old sub-ID in the target association group, avoiding the same sub-ID from being added to the target association group multiple times, thereby improving the accuracy of the detection.

[0074] S206: Detect the target association group based on a feature database to obtain a detection result, wherein the feature database includes feature information of the target object.

[0075] According to an embodiment of the present invention, the method for collecting the feature information in the feature database includes but is not limited to collecting the feature information through road condition data, and may also be collecting the feature information by registering the non-motor vehicle for license plate registration or registering the feature database.

[0076] According to an embodiment of the present invention, the feature information mainly refers to the basic features of the target object and other information, and the feature information can be image data and information data, frame image data, video data within a specific time period, or information data extracted from the target object. The feature information includes but is not limited to pictures, videos, text, audio and other forms. The content of the feature information of the target object includes but is not limited to the type, detection information and status information of the non-motor vehicle.

[0077] According to an embodiment of the present invention, when the target object refers to a non-motor vehicle, the target object may include various types of non-motor vehicles. The passenger capacity of different types of non-motor vehicles may be the same or different, so the type of non-motor vehicle is also a prerequisite for overload detection of the target association group. The detection information of the non-motor vehicle mainly includes the passenger capacity standards corresponding to different types of non-motor vehicles, that is, the relevant thresholds for detecting overload. The type and detection information of the non-motor vehicle can be set according to the specific situation and are not limited here.

[0078] According to one embodiment of the present invention, the status information of a non-motor vehicle is specifically: for example, the status information of a non-motor vehicle when it is not overloaded and the status information of a non-motor vehicle when it is overloaded. Here, non-motor vehicle overloading mainly refers to the number of people exceeding the number specified in the traffic rules.

[0079] It is worth mentioning that the status information of non-motor vehicles includes status information of different types of non-motor vehicles and status information of non-motor vehicles in different scenarios, wherein the status information includes information presented in the form of pictures, videos, texts, audios, etc. in the status scenario and information of people and vehicles as target association groups.

[0080] According to an embodiment of the present invention, the feature database may be updated, including but not limited to online update or offline update, that is, the feature information in the feature database may be increased, decreased or changed.

[0081] According to an embodiment of the present invention, referring to Figure 5 As shown, the step S206 also includes:

[0082] S2061, matching the types of the target objects in the target association group based on the feature database to obtain matching results, where the matching results include a first matching result and a second matching result;

[0083] S2062: Based on the matching result, detect the first matching result using a first reference threshold, and detect the second matching result using a second reference threshold to obtain a first detection result;

[0084] S2063: Based on the first detection result, classify and detect the target association group to obtain a second detection result.

[0085] According to an embodiment of the present invention, the step S2061 can be specifically implemented as: performing type matching on the primary ID of the target object of the target association group according to the feature database, that is, matching the specific type of the non-motor vehicle, wherein the type of the non-motor vehicle determines the number of passengers of the motor vehicle, so as to subsequently determine whether the non-motor vehicle has overloaded behavior. When the primary ID of the target object of the target association group can be matched with the type of the non-motor vehicle in the feature database, it is a first matching result; when the primary ID of the target object of the target association group does not match the type of the non-motor vehicle in the feature database, it is a second matching result.

[0086] According to an embodiment of the present invention, according to step S2062, the first matching result corresponds to the first reference threshold, that is, each type of non-motor vehicle corresponds to a first reference threshold for judging whether the non-motor vehicle is overloaded, and the first reference threshold is the maximum number of passengers of the non-motor vehicle within the scope specified by the traffic rules; when the matching result is the second matching result, that is, the main ID of the target object of the target association group does not match the non-motor vehicle type in the feature database, then there is no corresponding maximum number of passengers of the non-motor vehicle within the scope specified by the traffic rules, and the second reference threshold is corresponded by default. Preferably, the second reference threshold is 2, which is a value assigned by experience, and the number of passengers of most non-motor vehicles is 2. The second reference threshold can be specifically set, and a preferred value is provided here without being used as a limitation.

[0087] According to one embodiment of the present invention, the first matching result is detected using a first benchmark threshold to obtain a first detection result, wherein the first detection result specifically refers to: if the first matching result is greater than the first benchmark threshold, then it is judged that the probability of overload behavior occurring in the target association group is high; if the first matching result is less than or equal to the first benchmark threshold, then it is judged that the target association group is not overloaded.

[0088] According to one embodiment of the present invention, the second matching result is detected using a second reference threshold to obtain a first detection result, wherein the first detection result specifically refers to: if the second matching result is greater than the second reference threshold, then it is judged that the probability of overload behavior occurring in the target association group is high; if the second matching result is less than or equal to the second reference threshold, then it is judged that the target association group is not overloaded.

[0089] According to one embodiment of the present invention, according to step S2062, based on the first detection result, the target association group with a higher probability of overloading behavior is classified and detected, wherein the classification detection is mainly performed through a classification model, and mainly excludes the situation in which the sub-ID in the target association group is not the main ID, i.e., the driver or passenger of the non-motor vehicle, but other people who have been around the non-motor vehicle for a long time, and excludes the situation in which the sub-ID in the target association group is not the main ID, i.e., the driver or passenger of the non-motor vehicle, but some other misdetected objects.

[0090] It is worth mentioning that in step S204, multi-target tracking and association are performed on the first feature information in the 2D plane. Due to the limitations of the 2D plane viewing angle and the fact that the first feature information does not include 3D depth information, there is a high probability that the main ID and sub-ID in the target association group will be misdetected. Therefore, it is necessary to further classify and detect the target association group to exclude some situations where vehicles stay for a long time or travel for a long time around non-motor vehicles, and the sub-ID is not the main ID, i.e. the driver or passenger of the non-motor vehicle, but some other misdetected objects (such as goods), thereby improving the accuracy of the detection results.

[0091] S208: Update the feature database based on the detection result.

[0092] According to one embodiment of the present invention, in step S206, when the type of the target object in the target association group does not match the corresponding type in the feature database, the feature information of the target object in the target association group can be updated to the feature database, thereby expanding the feature information included in the feature database, and when the main ID of the target object, i.e., the non-motor vehicle, appears again, the behavior of the target association group can be effectively detected.

[0093] The method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0094] In the present embodiment, a data detection device is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the description has been made no further. As used below, the term "module" can implement the combination of software and / or hardware of the predetermined function. Although the device described in the following embodiments is preferably implemented with software, the implementation of hardware, or the combination of software and hardware is also possible and conceived.

[0095] According to another embodiment of the present invention, Figure 6 , a data detection device is provided, comprising: a first acquisition module 30, used to acquire target area data, detect the target area data through a detection model, and obtain first feature information of the target area, wherein the first feature information includes depth features of all target objects in the target area;

[0096] According to an embodiment of the present invention, the target area data refers to image data within the target area, wherein the image data includes but is not limited to picture data and video data, and the target area data is obtained by receiving and decoding data information transmitted by a data acquisition device. The data acquisition device may refer to a data acquisition device such as a camera or a video camera. In this embodiment, the image data is mainly obtained by decoding data information collected by the camera or the video camera. The target area can be set according to specific needs by setting the installation position of the data acquisition device.

[0097] According to an embodiment of the present invention, the detection model is a deep learning detection algorithm model, such as the YOLOX detection algorithm, which is pre-trained with the ImageNet dataset, and then fine-tuned with non-motor vehicles and human data in traffic scenes, and finally the detector outputs information such as coordinate frames, positions, and confidences of two categories of non-motor vehicles and human bodies. By detecting the target area data with the detection model, all object points of non-motor vehicles and humans in the target area data can be detected, and the feature information of non-motor vehicles and humans corresponding to the object points can be detected and output.

[0098] According to one embodiment of the present invention, the non-motor vehicles and humans are the target objects that need to be detected in the target area, wherein the first feature information refers to the depth features of all the target objects in the target area data, namely, non-motor vehicles and humans, and the depth features include but are not limited to features such as type, size, position, timestamp and confidence of non-motor vehicles, and features such as size, position, timestamp and confidence of humans.

[0099] The target association module 40 is used to perform multi-target association on the first feature information to establish a target association group; wherein the target association module 40 further includes:

[0100] A multi-target tracking module 41, configured to perform multi-target tracking on the first feature information to obtain second feature information, wherein the second feature information includes an identifier and a motion trajectory of the target object;

[0101] According to an embodiment of the present invention, the second feature information of the target object is obtained by performing multi-target tracking on the first feature information. That is, the second feature information of the target object is obtained by multi-target tracking processing based on the depth features of the target objects, i.e., non-motor vehicles and people, i.e., the type, size, position, timestamp, and confidence of the non-motor vehicles, and the size, position, timestamp, and confidence of people, wherein the second feature information includes the IDs and motion trajectories of all the target objects in the target area.

[0102] According to one embodiment of the present invention, the first feature information is tracked by using the IouTrack tracker to assign a unique identifier, i.e., the ID of the target object, to the target object and form a motion track of the target object represented by each ID. In other embodiments provided by the present invention, other multi-target tracking algorithms can also be used to perform multi-target tracking on the first feature information. As long as the same function can be achieved, the present invention is applicable and is not limited here.

[0103] The association processing module 42 is used to perform association processing on the second feature information to obtain the target association group.

[0104] According to an embodiment of the present invention, due to the large occlusion of the human body on the non-motor vehicle and the influence of the installation positions of different data acquisition devices, continuous detection is difficult and the continuity of the tracked ID is difficult to ensure, that is, it is impossible to form an effective and continuous motion trajectory. Therefore, in this embodiment, it is necessary to perform association processing on the second feature information after performing multi-target tracking processing on the first feature information, and improve the accuracy of detection by obtaining a target association group.

[0105] According to an embodiment of the present invention, the association processing module 42 specifically includes:

[0106] A target association area setting unit 421, configured to set a target association area based on the first feature information and the second feature information;

[0107] The association screening unit 422 is used to perform association screening on the target object based on the target association area to obtain the target association group;

[0108] The target association group updating unit 423 is configured to update the target association group to obtain a new target association group.

[0109] According to one embodiment of the present invention, based on the second characteristic information, the ID of the non-motor vehicle is set as the main ID, and an association threshold is set. The association threshold may refer to a distance threshold. The target association area is formed based on the association threshold with the main ID as the center.

[0110] According to an embodiment of the present invention, association conditions are set with the main ID as the center, the target objects in the target association area are screened according to the association conditions, and the target objects that meet the association conditions are set as sub-IDs corresponding to the main ID.

[0111] According to one embodiment of the present invention, the association conditions include time conditions, trajectory similarity conditions, direction conditions and speed conditions. When the target object in the target association area meets all of the above-mentioned association conditions, the target object is set to the sub-ID corresponding to the main ID.

[0112] According to an embodiment of the present invention, the association condition may include a trajectory similarity condition. The similarity between the human body trajectory and the non-motor vehicle trajectory is calculated through a trajectory similarity matching algorithm model, and a trajectory similarity threshold is set. For example, the trajectory similarity threshold is set to 90%. Based on the first feature information and the second feature information, that is, the deep feature of the target object and the ID and corresponding trajectory of the target object, the trajectory similarity is calculated. When the trajectory similarity is greater than (or greater than or equal to) 90%, the target object meets the trajectory similarity condition. When the target object does not meet the trajectory similarity condition, the association screening is no longer performed. The trajectory similarity matching algorithm model may be a Fréchet distance algorithm, or other algorithms with similar functions, which are not limited here. It is worth mentioning that the trajectory similarity threshold can be set according to specific circumstances and needs.

[0113] According to an embodiment of the present invention, the association condition may include a time condition, and the time condition may be specifically set as a time threshold, for example, the time threshold is set to 10s, and the duration of similar trajectories of the human body and non-motor vehicle is calculated based on the first feature information and the second feature information, i.e., the depth feature of the target object and the ID and corresponding trajectory of the target object. If the duration is greater than (or greater than or equal to) 10s, the target object satisfies the time condition. If the duration is less than 10s, it is judged that the time condition is not met, and the association screening is no longer performed. It is worth mentioning that the time threshold can be set specifically according to specific circumstances and needs.

[0114] According to an embodiment of the present invention, the association condition may include a direction condition, and the direction condition may be specifically set to a motion direction deviation threshold, for example, the motion direction deviation threshold is set to 5%, and based on the first feature information and the second feature information, i.e., the depth feature of the target object and the ID and corresponding trajectory of the target object, the motion direction deviation of similar trajectories of the human body and non-motor vehicle is calculated. If the motion direction deviation is less than (or less than or equal to) 5%, the target object satisfies the direction condition. If the motion direction deviation is greater than 5%, it is determined that the direction condition is not met, and the association screening is no longer performed. It is worth mentioning that the motion direction deviation threshold can be set specifically according to specific circumstances and needs.

[0115] According to an embodiment of the present invention, the association condition may include a speed condition, and the speed condition may be specifically set to a motion speed deviation threshold, for example, the motion speed deviation threshold is set to 5%, and based on the first feature information and the second feature information, i.e., the depth feature of the target object and the ID and corresponding trajectory of the target object, the motion speed deviation of similar trajectories of the human body and non-motor vehicle is calculated. If the motion speed deviation is less than (or less than or equal to) 5%, the target object satisfies the speed condition. If the motion speed deviation is greater than 5%, it is determined that the speed condition is not met, and the association screening is no longer performed. It is worth mentioning that the motion speed deviation threshold can be specifically set according to specific circumstances and needs.

[0116] In some other embodiments of the present invention, the association conditions may include other conditions besides the time condition, the trajectory similarity condition, the direction condition and the speed condition, that is, the conditions may be increased or decreased, and the association conditions may be set according to specific requirements. The above-listed association conditions are not intended to be limiting.

[0117] In some other embodiments of the present invention, the association condition may also include only some of the time condition, the trajectory similarity condition, the direction condition and the speed condition, that is, the conditions may be screened out. The association condition may be set according to specific requirements. The above-listed association conditions are not intended to be limiting.

[0118] According to one embodiment of the present invention, with the main ID as the center, when the target object in the target association area meets the association condition set when performing association screening, the target object of the association condition is set to the sub-ID corresponding to the main ID, and the main ID and the sub-ID constitute the target association group.

[0119] According to one embodiment of the present invention, the target area is continuously associated with the screening. For different sub-IDs that meet the association conditions after the association screening, if it is a new sub-ID, the depth features of the human body represented by the sub-ID are extracted and matched with the human body features of the target association group. If the match is successful, the trajectory of the new sub-ID is updated to the corresponding sub-ID in the target association group. If the match fails, the sub-ID is included in the target association group as a new sub-ID. That is, if it is an old sub-ID, the motion trajectory of the corresponding sub-ID in the target association group is updated. The above implementation process updates the target association group to obtain a new target association group.

[0120] It is worth mentioning that the sub-ID of the target association group may disappear briefly in the target area due to reasons such as being blocked. When the blockage disappears, the sub-ID reappears. Therefore, it is necessary to avoid adding the same sub-ID to the target association group. Otherwise, two identical sub-IDs will appear in the target association group, which will have a significant impact on the subsequent non-motor vehicle overload detection results and greatly reduce the accuracy of the detection results. This situation can be effectively detected by the target association group update unit 423. When the sub-ID appears again, the trajectory of the new sub-ID is used to update the corresponding old sub-ID in the target association group, avoiding the same sub-ID from being added to the target association group multiple times, thereby improving the accuracy of the detection.

[0121] The detection module 50 detects the target association group based on a feature database to obtain a detection result, wherein the feature database includes feature information of the target object.

[0122] According to an embodiment of the present invention, the method for collecting the feature information in the feature database includes but is not limited to collecting the feature information through road condition data, and may also be collecting the feature information by registering the non-motor vehicle for license plate registration or registering the feature database.

[0123] According to an embodiment of the present invention, the feature information mainly refers to the basic features of the target object and other information, and the feature information can be image data and information data, frame image data, video data within a specific time period, or information data extracted from the target object. The feature information includes but is not limited to pictures, videos, text, audio and other forms. The content of the feature information of the target object includes but is not limited to the type, detection information and status information of the non-motor vehicle.

[0124] According to an embodiment of the present invention, when the target object refers to a non-motor vehicle, the target object may include various types of non-motor vehicles. The passenger capacity of different types of non-motor vehicles may be the same or different, so the type of non-motor vehicle is also a prerequisite for overload detection of the target association group. The detection information of the non-motor vehicle mainly includes the passenger capacity standards corresponding to different types of non-motor vehicles, that is, the relevant thresholds for detecting overload. The type and detection information of the non-motor vehicle can be set according to the specific situation and are not limited here.

[0125] According to one embodiment of the present invention, the status information of a non-motor vehicle is specifically: for example, the status information of a non-motor vehicle when it is not overloaded and the status information of a non-motor vehicle when it is overloaded. Here, non-motor vehicle overloading mainly refers to the number of people exceeding the number specified in the traffic rules.

[0126] It is worth mentioning that the status information of non-motor vehicles includes status information of different types of non-motor vehicles and status information of non-motor vehicles in different scenarios, wherein the status information includes information presented in the form of pictures, videos, texts, audios, etc. in the status scenario and information of people and vehicles as target association groups.

[0127] According to an embodiment of the present invention, the feature database may be updated, including but not limited to online update or offline update, that is, the feature information in the feature database may be increased, decreased or changed.

[0128] According to an embodiment of the present invention, the detection module 50 further includes:

[0129] A matching unit 51, configured to match the type of the target object in the target association group based on the feature database to obtain a matching result, wherein the matching result includes a first matching result and a second matching result;

[0130] A first detection unit 52 is used to detect the first matching result using a first reference threshold and detect the second matching result using a second reference threshold based on the matching result to obtain a first detection result;

[0131] The classification detection unit 53 is used to perform classification detection on the target association group based on the first detection result to obtain a second detection result.

[0132] According to an embodiment of the present invention, the main ID of the target object of the target association group is matched according to the feature database, that is, the specific type of the non-motor vehicle is matched, wherein the type of the non-motor vehicle determines the number of passengers of the motor vehicle, so as to subsequently determine whether the non-motor vehicle has overloaded behavior. When the main ID of the target object of the target association group can be matched with the type of the non-motor vehicle in the feature database, it is a first matching result; when the main ID of the target object of the target association group does not match the type of the non-motor vehicle in the feature database, it is a second matching result.

[0133] According to an embodiment of the present invention, the first matching result corresponds to the first reference threshold, that is, each type of non-motor vehicle corresponds to a first reference threshold for judging whether the non-motor vehicle is overloaded, and the first reference threshold is the maximum number of passengers of the non-motor vehicle within the scope specified by the traffic rules; when the matching result is the second matching result, that is, the main ID of the target object of the target association group does not match the type of non-motor vehicle in the feature database, there is no corresponding maximum number of passengers of the non-motor vehicle within the scope specified by the traffic rules, and the second reference threshold is corresponded by default. Preferably, the second reference threshold is 2, which is a value assigned by experience, and the number of passengers of most non-motor vehicles is 2. The second reference threshold can be specifically set, and a preferred value is provided here without being used as a limitation.

[0134] According to one embodiment of the present invention, the first matching result is detected using a first benchmark threshold to obtain a first detection result, wherein the first detection result specifically refers to: if the first matching result is greater than the first benchmark threshold, then it is judged that the probability of overload behavior occurring in the target association group is high; if the first matching result is less than or equal to the first benchmark threshold, then it is judged that the target association group is not overloaded.

[0135] According to one embodiment of the present invention, the second matching result is detected using a second reference threshold to obtain a first detection result, wherein the first detection result specifically refers to: if the second matching result is greater than the second reference threshold, then it is judged that the probability of overload behavior occurring in the target association group is high; if the second matching result is less than or equal to the second reference threshold, then it is judged that the target association group is not overloaded.

[0136] According to one embodiment of the present invention, based on the first detection result, classification detection is performed on the target association group with a higher probability of overloading behavior, wherein the classification detection is mainly performed through a classification model, and mainly excludes the situation in which the sub-ID in the target association group is not the main ID, i.e., the driver or passenger of the non-motor vehicle, but other people who have been around the non-motor vehicle for a long time, and excludes the situation in which the sub-ID in the target association group is not the main ID, i.e., the driver or passenger of the non-motor vehicle, but some other misdetected objects.

[0137] It is worth mentioning that the target association module 40 specifically performs multi-target tracking and association on the first feature information in the 2D plane. Due to the limitations of the 2D plane viewing angle and the fact that the first feature information does not include 3D depth information, there is a high probability that the main ID and sub-ID in the target association group will be misdetected. Therefore, it is necessary to re-classify the target association group to exclude some situations where vehicles stay for a long time or travel for a long time around non-motor vehicles, and the sub-ID is not the main ID, i.e. the driver or passenger of the non-motor vehicle, but some other misdetected objects (such as goods), thereby improving the accuracy of the detection results.

[0138] The updating module 60 updates the feature database based on the detection result.

[0139] According to one embodiment of the present invention, when the type of the target object in the target association group does not match the corresponding type in the feature database, the feature information of the target object in the target association group can be updated to the feature database, thereby expanding the feature information included in the feature database. When the main ID of the target object, i.e., the non-motor vehicle, appears again, the behavior of the target association group can be effectively detected.

[0140] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0141] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0142] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0143] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0144] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0145] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data detection method, It is characterized in that include: Acquire target area data, detect the target area data through a detection model, and obtain first feature information of the target area, wherein the first feature information includes depth features of all target objects in the target area; Performing multi-target association on the first feature information to establish a target association group; Based on a feature database, the target association group is detected to obtain a detection result, wherein the feature database includes feature information of the target object; Perform multi-target association on the first feature information to establish a target association group, including: setting a target association area based on the first feature information and the second feature information, wherein the second feature information includes the identifier and motion trajectory of the target object; based on the target association area and the association condition, performing association screening on the target object to obtain the target association group, wherein the association condition is centered on a main identifier and is determined based on the first feature information and the second feature information, the main identifier is an identifier of a non-motor vehicle determined based on the second feature information, and the target objects include the non-motor vehicle; updating the target association group to obtain a new target association group.

2. The method according to claim 1, It is characterized in that Performing multi-target association on the first feature information to establish a target association group includes: Performing multi-target tracking on the first feature information to obtain the second feature information; Perform association processing on the second characteristic information to obtain the target association group.

3. The method according to claim 1, It is characterized in that Based on a feature database, the target association group is detected to obtain a detection result, wherein the feature database includes feature information of the target object, including: Based on the feature database, matching the types of the target objects in the target association group to obtain matching results, wherein the matching results include a first matching result and a second matching result; Based on the matching result, the first matching result is detected using a first reference threshold, and the second matching result is detected using a second reference threshold to obtain a first detection result; Based on the first detection result, the target association group is classified and detected to obtain a second detection result.

4. The method according to claim 1, It is characterized in that Also includes: Based on the detection result, the feature database is updated.

5. A data detection device, It is characterized in that include: A first acquisition module is used to acquire target area data, detect the target area data through a detection model, and obtain first feature information of the target area, wherein the first feature information includes depth features of all target objects in the target area; A target association module, used for performing multi-target association on the first feature information to establish a target association group; A detection module detects the target association group based on a feature database to obtain a detection result, wherein the feature database includes feature information of the target object; The target association module is also used to set a target association area based on the first feature information and the second feature information, wherein the second feature information includes the identifier and motion trajectory of the target object; based on the target association area and the association condition, the target object is associated and screened to obtain the target association group, wherein the association condition is centered on a main identifier and is determined based on the first feature information and the second feature information, the main identifier is an identifier of a non-motor vehicle determined based on the second feature information, and the target object includes the non-motor vehicle; the target association group is updated to obtain a new target association group.

6. The device according to claim 5, It is characterized in that The target association module includes: A multi-target tracking module, used to perform multi-target tracking on the first feature information to obtain the second feature information; An association processing module is used to perform association processing on the second feature information to obtain the target association group.

7. The device according to claim 6, It is characterized in that The detection module comprises: A matching unit, configured to match the type of the target object in the target association group based on the feature database to obtain a matching result, wherein the matching result includes a first matching result and a second matching result; A first detection unit, configured to detect the first matching result based on the matching result using a first reference threshold, and detect the second matching result using a second reference threshold, to obtain a first detection result; A classification detection unit is used to perform classification detection on the target association group based on the first detection result to obtain a second detection result.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 4 when executed.

9. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 4.

Images