Swarm intelligence object detection method, storage medium and processor

Abstract

The application discloses a swarm intelligence object detection method, a storage medium and a processor. The method comprises the following steps: acquiring a target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; and determining whether the first target object is qualified based on a target parameter of the target model. The application solves the technical problem of the accuracy of object detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer, in particular to an object detection method based on swarm intelligence, a storage medium and a processor. BACKGROUND

[0002] At present, when detecting an object, it can be to detect an occupation of a road, which is usually realized by human resources, and the efficiency is low.

[0003] In addition, the object can also be detected based on a target detection algorithm, but it can only identify the track and time length of a suspected object, and the detection ability of whether the object is truly unqualified is still relatively insufficient, and it is difficult to judge the behavior of the object, which is easy to cause false positives, thereby there is a technical problem of low accuracy in detecting the object.

[0004] At present, no effective solution has been proposed for the above problems. SUMMARY

[0005] Embodiments of the present application provide an object detection method based on swarm intelligence, a storage medium and a processor to at least solve the technical problem of the accuracy of detecting an object.

[0006] According to an aspect of an embodiment of the present application, an object detection method based on swarm intelligence is provided. The method can include: obtaining a target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; and determining whether the first target object is qualified based on a target parameter of the target model.

[0007] According to an aspect of an embodiment of the present application, another object detection method based on swarm intelligence is also provided. The method can include: in response to a video input instruction acting on an interactive interface, obtaining a target video; and in response to a detection instruction acting on the interactive interface, outputting a detection result, wherein the detection result is used to represent whether a first target object in the target video is qualified, and is determined based on a target parameter of a target model, the target model is trained by target data in the target video, and the target data is used to represent behavior information of the first target object.

[0008] According to an aspect of some embodiments of the present application, there is also provided another method for object detection based on swarm intelligence. The method can include: obtaining a target image by calling a first interface, wherein the first interface comprises a first parameter, and a parameter value of the first parameter is the target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; determining whether the first target object is qualified based on a target parameter of the target model, to obtain a detection result; and outputting the detection result by calling a second interface, wherein the second interface comprises a second parameter, and a parameter value of the second parameter is the detection result.

[0009] According to an aspect of some embodiments of the present application, there is also provided a device for object detection based on swarm intelligence. The device can include: a first obtaining unit configured to obtain a target image; a first extracting unit configured to extract target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; a first training unit configured to train a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; and a first determining unit configured to determine whether the first target object is qualified based on a target parameter of the target model.

[0010] According to an aspect of some embodiments of the present application, there is also provided another device for object detection based on swarm intelligence. The device can include: a second obtaining unit configured to obtain a target video in response to a video input instruction acting on an interactive interface; and an output unit configured to output a detection result in response to a detection instruction acting on the interactive interface, wherein the detection result is used to represent whether a first target object in the target video is qualified, and is determined based on a target parameter of a target model, the target model being trained based on target data in the target video, and the target data being used to represent behavior information of the first target object.

[0011] According to an aspect of some embodiments of the present application, there is also provided another device for object detection based on swarm intelligence. The device can include: a first calling unit configured to obtain a target image by calling a first interface, wherein the first interface comprises a first parameter, and a parameter value of the first parameter is the target image; a second extracting unit configured to extract target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; a second training unit configured to train a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; a second determining unit configured to determine whether the first target object is qualified based on a target parameter of the target model, to obtain a detection result; and a second calling unit configured to output the detection result by calling a second interface, wherein the second interface comprises a second parameter, and a parameter value of the second parameter is the detection result.

[0012] According to an aspect of the embodiments of the present application, a computer readable storage medium is also provided. The computer readable storage medium includes a stored program, wherein the program, when executed by a processor, controls the device where the computer readable storage medium is located to perform the object detection method of the swarm intelligence.

[0013] According to an aspect of the embodiments of the present application, a processor is also provided. The processor is configured to execute a program, wherein the program, when executed, performs the object detection method of the swarm intelligence.

[0014] According to an aspect of the embodiments of the present application, an object detection system of the swarm intelligence is also provided. The system can include: a processor; a memory connected to the processor, configured to provide the processor with instructions for processing the following processing steps: obtaining a target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; determining whether the first target object is qualified based on a target parameter of the target model.

[0015] In the embodiments of the present application, the target image is obtained; the target data is extracted from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; the target model is trained based on the target data, wherein the target model is used to simulate a motion process of the first target object; and whether the first target object is qualified is determined based on a target parameter of the target model. The target data extracted from the target image in the embodiments can determine the behavior information of the first target object, and then the target model is trained based on the target data to determine the motion process of the target object, and then whether the first target object is qualified is inferred based on the target parameter of the target model, thereby avoiding the problem that only the track and time length of the suspected object can be identified, and the detection ability of whether the object is truly unqualified is still relatively insufficient, which is easy to cause false positives, thereby solving the technical problem of the accuracy of detecting the object, and achieving the technical effect of improving the accuracy of detecting the object. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the present application and constitute a part of this application, illustrate certain illustrative embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0017] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing the object detection method of the swarm intelligence according to an embodiment of the present application;

[0018] Figure 2 is a flowchart of the object detection method of the swarm intelligence according to an embodiment of the present application;

[0019] Figure 3 is a flow chart of another swarm intelligence object detection method according to an embodiment of the present application;

[0020] Figure 4 is a flow chart of another swarm intelligence object detection method according to an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of a violation operation detection output result according to an embodiment of the present application;

[0022] Figure 6 is a flow chart of a violation operation detection method according to an embodiment of the present application;

[0023] Figure 7 is a schematic diagram of a swarm intelligence object detection device according to an embodiment of the present application;

[0024] Figure 8 is a schematic diagram of another swarm intelligence object detection device according to an embodiment of the present application;

[0025] Figure 9 is a schematic diagram of another swarm intelligence object detection device according to an embodiment of the present application;

[0026] Figure 10 is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the personnel in the art better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] First, some of the nouns or terms that appear in the description of the embodiments of the present application are applicable to the following explanations:

[0030] Swarm Intelligence, Swarm Intelligence is a kind of machine intelligence method that solves super large scale complex problems by gathering group wisdom, which provides new technology and means for solving super large scale complex problems that are difficult to solve by traditional methods, and has been widely used in traffic, crowd computing, software development and other fields. For specific application scenarios, domestic and foreign scholars have designed various swarm intelligence methods from the perspectives of individual evaluation mechanism construction, individual coding and decoding strategy, and group organization architecture;

[0031] Crowd Simulation, is a process for simulating the movement of a large number of entities or characters, used for crisis training, building and city planning, and evacuation simulation, and can also be used for virtual scene creation in movies or video games;

[0032] Object Detection, a branch of computer technology closely related to computer vision and image processing, aims to detect specific semantic target entities in digital images and videos, such as detecting people, buildings, cars, etc., and outputting a tightly wrapped target entity rectangular frame as a result. Target detection has applications in many computer vision fields such as image retrieval and video surveillance;

[0033] Computer Vision, a science that studies how to make machines see, further, refers to replacing human eyes with cameras and computers to identify, track and measure targets, etc. Machine vision, and further image processing with computers to become more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data.

[0034] Embodiment 1

[0035] According to the embodiments of the present application, an embodiment of a swarm intelligence object detection method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0036] The method embodiment provided by the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a swarm intelligence object detection method according to an embodiment of the present application. As shown in Figure 1 The computer terminal 10 (or mobile device 10) can include one or more processors 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or fewer components than those shown in Figure 1 or have a different configuration than that shown in Figure 1 .

[0037] It should be noted that the one or more processors 102 and / or other data processing circuits described above can be referred to herein as "data processing circuits" in general. The data processing circuit can be embodied in whole or in part as software, hardware, firmware or any combination thereof. In addition, the data processing circuit can be a single independent processing module, or any one of the other elements combined into the computer terminal 10 (or mobile device) in whole or in part. As referred to in the embodiments of the present application, the data processing circuit serves as a processor to control (for example, selection of a variable resistance terminal path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage means corresponding to the swarm intelligence object detection method of the application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the swarm intelligence object detection method of the application program described above. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, the memory 104 can further include a memory remotely disposed with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0039] The transmission device 106 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network interface controller (NIC) that can connect to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.

[0040] The display can be a liquid crystal display (LCD) that is touch screen, for example, which can enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0041] It should be noted that in some alternative embodiments, the above-mentioned Figure 1 The computer device (or mobile device) can include hardware elements (including circuitry), software elements (including computer code stored on a computer readable medium), or a combination of both hardware and software elements. It should be noted that in some alternative embodiments, the functions described herein can be implemented on different systems involving software, firmware, hardware, or any combination of the three. Figure 1 is merely one example of a particular implementation, and is intended to illustrate the types of components that can be present in the above-mentioned computer device (or mobile device).

[0042] In the above-mentioned operating environment, the present application provides a method for object detection of swarm intelligence as shown in Figure 1 The method for object detection of swarm intelligence can be executed by the mobile terminal of the above-mentioned embodiment. Figure 2 The method for object detection of swarm intelligence can be executed by the mobile terminal of the above-mentioned embodiment. Figure 1 The method for object detection of swarm intelligence can be executed by the mobile terminal of the above-mentioned embodiment.

[0043] Figure 2 is a flowchart of a method for object detection of swarm intelligence according to an embodiment of the present application. As shown in Figure 2 The method can include the following steps:

[0044] In step S202, a target image is obtained.

[0045] In the above-mentioned technical solution of step S202 of the present application, the target image can be an image obtained by photographing a target region in a target scene, wherein the target scene can be a city management scene, and the target region can be a region that needs to be renovated, such as a road region, a region where a store is located, etc., which is not limited here.

[0046] The embodiment obtains a target image, which can be obtained by an image acquisition device deployed in the target region, and can be a people flow plane trajectory diagram, and can reflect the capability of the system for people flow detection and tracking. Since tracking and modeling simulation both need data for a period of time, the target image of the embodiment can be a video image, which presents a certain picture. The image acquisition device can be a camera, which is not specifically limited here, so that the target region can also be referred to as a camera region. Alternatively, the embodiment obtains an input image frame.

[0047] In step S204, target data is extracted from the target image, wherein the target data is used to represent behavior information of the first target object to be detected.

[0048] In the technical solution provided by the above step S204 of the present application, after the target image is obtained, target data is extracted from the target image, wherein the target image can include a first target object to be recognized, which can be a large number of entities or roles in the target region. Alternatively, the first target object is a pedestrian, which can be a crowd, such as a crowd around a road. Then, the target data can be used to represent behavior information of the first target object, which is used to represent the behavior of the first target object, such as the behavior information of the crowd, which is used to represent the behavior of the crowd. The behavior can be a swarm intelligence behavior, so as to achieve the purpose of judging the behavior of the first target object.

[0049] Alternatively, when there is an unqualified behavior in the target scene, the behavior information of the first target object in the target region will change, such as the change of the movement mode, and the target data will change obviously, such as when there is a violation of the operating behavior in the target scene, the pedestrian flow pattern will change obviously, which can be gathering or detouring, etc.

[0050] Alternatively, the embodiment can obtain historical data obtained by detecting the target region, and can extract target data from the target image in combination with the historical data.

[0051] Alternatively, the embodiment can pre-train a first target detection model, such as a target detection and tracking model, which can be used to identify the first target object in the target image, so as to obtain target data, which can include a people flow trajectory.

[0052] Alternatively, the above first target detection model of the embodiment can be obtained by training data labeled by artificial before going online (which can be the same image as the classification model training mentioned later, or other public data sets or self-collected data, and the labeled content is different).

[0053] Step S206, training the target model based on the target data, wherein the target model is used to simulate the movement process of the first target object.

[0054] In the technical solution provided in the above step S206 of the present application, after the target data is extracted from the target image, the target data can be used as the input of the training of the target model, and then the target model is trained based on the target data, which can be by collecting the target data in different time periods / scenes to train the target model. Optionally, the embodiment can construct the target model in a predetermined manner, wherein the predetermined manner can be a swarm intelligence technology and a crowd simulation technology. The swarm intelligence can be a machine intelligence method for solving super-large-scale complex problems by converging swarm intelligence. The crowd simulation is a process of simulating the movement of the target object. Therefore, the target model of the embodiment can be a swarm intelligence crowd simulation model, which is used to simulate the movement process of the first target object, such as simulating the flow direction of the crowd in the target area, so that the embodiment achieves the purpose of modeling (swarm intelligence people flow modeling) the flow direction of the crowd in the target area. The swarm intelligence crowd simulation model can also be referred to as a crowd simulation model, a crowd model, or a swarm intelligence model.

[0055] Step S208, determining whether the first target object is qualified based on the target parameter of the target model.

[0056] In the technical solution provided in the above step S208 of the present application, after the target model is trained based on the target data, whether the first target object is qualified can be determined based on the target parameter of the target model.

[0057] In this embodiment, the target model includes the target parameter. The target parameter of the target model can be input into a pre-trained classification model, and the target parameter of the target model is analyzed by the classification model to determine whether the first target object is qualified, to obtain a detection result, from which the flow direction rule and the cause of the first target object can be analyzed.

[0058] Optionally, the above classification model of the embodiment can output the confidence of whether there is unqualified behavior in the target scene, and can output a binary classification (there is / there is no irregular operation) label.

[0059] Optionally, if it is determined based on the target parameter of the target model that the behavior of the first target object in the target area is an irregular operation behavior, it can be determined that the first target object is unqualified, which means that it is an irregular operation target object, that is, it is determined that there is an irregular operation behavior in the target area, so that the above unqualified can refer to irregular operation.

[0060] Optionally, the embodiment locates the object in the target area that exists in the illegal operation; if it is determined based on the target parameter of the target model that the behavior of the first target object in the target area does not belong to the illegal operation behavior, it can be determined that the first target object is qualified, which means that the first target object does not generate the illegal operation behavior, that is, it is determined that there is no illegal operation behavior in the target area. Thus, the embodiment can realize a crowd modeling method of swarm intelligence, infer whether there is an object that generates illegal operation behavior in the target area from the flow trajectory pattern, thereby improving the accuracy of detecting the object.

[0061] Optionally, the embodiment can analyze the behavior rule of the first target object in the target scene by using the target model, which can be to analyze the change rule of the swarm intelligence in the target area to determine whether the first target object is qualified.

[0062] After determining whether the first target object is qualified based on the target parameter of the target model, the embodiment can report the unqualified first target object, achieving the purpose of accurately reporting the detection result of the first target object.

[0063] Through the above steps S202 to S208, the target image is obtained; the target data is extracted from the target image, wherein the target data is used to represent the behavior information of the first target object to be identified; the target model is trained based on the target data, wherein the target model is used to simulate the motion process of the first target object; and whether the first target object is qualified is determined based on the target parameter of the target model. That is, the behavior information of the first target object can be determined by the target data extracted from the target image, and then the target model is trained by using the target data to determine the motion process of the target object, and then whether the first target object is qualified is inferred based on the target parameter of the target model, which avoids only identifying the trajectory and time length of the suspected object, and the detection ability of whether the object is truly unqualified is still relatively insufficient, which is easy to cause false positives, thereby solving the technical problem of the accuracy of detecting the object, and achieving the technical effect of improving the accuracy of detecting the object.

[0064] The above method of the embodiment will be further introduced below.

[0065] As an optional implementation, the method further includes: determining that a second target object exists in the target image, wherein a confidence of the second target object is greater than a first threshold; and determining whether the first target object is qualified based on the parameter of the target model, including: determining whether the first target object is qualified based on the second target object and the target parameter.

[0066] In this embodiment, the target image can be detected by using the target detection method to obtain that the second target object exists in the target image, and the confidence of the second target object is greater than the first threshold. The first threshold can be a threshold for measuring the confidence of the second target object. Optionally, the second target object can be detected by using the second target detection model in the target image, and the second target object is an operating object. As a category, the second target object can be a back basket, a desktop, a box, a frame, a basket, a bucket, a stall cloth, and the like.

[0067] Optionally, the confidence can be understood as the product of the probability of the existence of the operating object such as the back basket and the desktop and the "unqualified" of the operating object.

[0068] The embodiment can determine whether the first target object is qualified based on the second target object and the target parameter. The target parameter of the target model can be analyzed based on the second target object, so as to further analyze the role of the second target object in the target scene.

[0069] The embodiment can analyze whether the first target object is qualified from two dimensions of detecting the first target object and determining the target parameter of the target model by combining the target detection method and the crowd modeling method based on swarm intelligence. For example, by combining the two dimensions of detecting the suspected operating object and the change of the swarm intelligence law in the region, it can be determined whether there is a violation of the operating behavior in the target region, so as to improve the accuracy of detecting the object.

[0070] The embodiment can analyze the target parameter of the target model based on the detection of the second target object in the target image, and obtain the target model by modeling the target data in the target region for representing the behavior information of the first target object. The auxiliary information for determining whether the first target object is qualified is provided, and the limitation that it is difficult to determine whether the object is qualified based on the computer vision method is avoided. By introducing the swarm intelligence behavior analysis, the accuracy of detecting the object can be improved. For example, by introducing the swarm intelligence behavior analysis, the limitation that it is difficult to determine the operating behavior based on the computer vision method is avoided, and the purpose of effectively reducing the false positive rate of the violation of the operating detection is achieved.

[0071] Optionally, the embodiment can execute the determination of the existence of the second target object in the target image and the training of the target model based on the target data in parallel to save time, or in series to reduce threads.

[0072] Optionally, the second target detection model of the embodiment for determining the presence of the second target object in the target image can be trained by training a target detection model that simultaneously includes detection of the second target object and identification of the first target object, thereby achieving the purpose of saving computing time.

[0073] Optionally, the second target detection model of the embodiment can also increase or integrate sub-categories, such as splitting boxes, frames, baskets, and barrels into two categories according to square and round shapes, or unifying rear hoppers and table surfaces into a table surface category, and the like.

[0074] Optionally, the second target detection model of the embodiment can be obtained by training through manually annotated data (which can be images used for training the classification model mentioned later, or other public data sets or self-collected data, with different annotation contents) before going online.

[0075] As an optional implementation, determining whether the first target object is qualified based on the second target object and the target parameter includes: determining a first region corresponding to an unqualified behavior in the behavior information based on the target parameter, wherein a probability of the unqualified behavior occurring in the first region is greater than a second threshold, and a region corresponding to the target image includes the first region; and determining whether the first target object is qualified based on the first region and a second region corresponding to the second target object.

[0076] In this embodiment, when determining whether the first target object is qualified based on the second target object and the target parameter, the target parameter can be input into a pre-trained classification model, the target parameter is processed by the classification model, and a first region corresponding to an unqualified behavior in the behavior information is output, wherein the unqualified behavior can be a rule violation behavior, and the first region can be a region where the rule violation behavior can exist.

[0077] This embodiment can determine a second region corresponding to the second target object, which can be a region where the second target object is located, such as a suspected operating object, and the second region can be a region where the suspected operating object is located. After determining the first region corresponding to the unqualified behavior and the second region where the second target object is located, whether the first target object is qualified can be determined based on the first region and the second region.

[0078] As an optional implementation, determining whether the first target object is qualified based on the first region and the second region corresponding to the second target object includes: screening out unqualified first target objects from the second region based on the first region.

[0079] The embodiment can utilize the first region corresponding to the unqualified behavior to filter the second region where the second target object is located, and screen out the unqualified first target object. That is, for the second region where the second target object is located, the result of not being in the first region corresponding to the unqualified behavior is filtered, and the unqualified first target object is returned, which can effectively reduce the false positive rate of illegal operation detection.

[0080] For example, the embodiment can utilize the region where illegal operation may exist to filter the region where the suspected operating object is located, and screen out the illegal operation target. That is, for the region where the obtained suspected operating object is located, the result of not being in the region where illegal operation may exist is filtered out, and then the illegal operation target is returned. For example, the result of not being in the region where illegal operation may exist is a pile of sundries, a truck / tricycle parked due to road congestion, and the like. In this way, the false positives caused by the pile of sundries, the truck / tricycle parked due to road congestion, and the like in the target scene can be filtered out, and the accuracy of the algorithm alarm is improved.

[0081] As an optional implementation, the method further includes: determining a confidence of the unqualified behavior in the behavior information based on the target parameter; and determining whether the first target object is qualified based on the first region and a second region corresponding to the second target object, including: determining that the confidence is greater than a third threshold value, and then determining whether the first target object is qualified based on the first region and the second region.

[0082] In the embodiment, the target parameter can be input into a pre-trained classification model, the target parameter is processed by the classification model, and a confidence that the unqualified behavior exists in the target region is obtained. The confidence can also be referred to as a confidence score. Optionally, if the unqualified behavior does not exist, the confidence can be 0. The embodiment can determine whether the confidence is greater than a third threshold value. If it is determined that the confidence is greater than the third threshold value, whether the first target object is qualified can be determined based on the first region and the second region. For example, in the second region where the second target object is located, the result of not being in the first region corresponding to the unqualified behavior is filtered, and the unqualified first target object is returned.

[0083] Optionally, the embodiment can reduce false detection even if the first region where the unqualified behavior occurs is not estimated by the target parameter of the target model, but only determined according to the confidence that the unqualified behavior exists. The first region where the unqualified behavior occurs is estimated by the target parameter of the target model, and whether the first target object is qualified is determined based on the first region and the second region, which can further improve the accuracy of detecting the first target object.

[0084] As an optional implementation, the first area corresponding to the unqualified behavior in the behavior information is determined based on the target parameter, including: converting the target parameter into a target vector; determining the first area based on the target vector.

[0085] In this embodiment, when the first area corresponding to the unqualified behavior in the behavior information is determined based on the target parameter, the plurality of target parameters of the target model can be converted into a target vector first, such as being composed into a target vector, and then the target vector is input into the classification model, the target vector is analyzed by the classification model, and the first area corresponding to the unqualified behavior is output, that is, the area where there may be illegal operation is output.

[0086] As an optional implementation, the method further includes: in response to a model pre-training instruction, obtaining labeled data; training the classification model based on the labeled data; and determining whether the first target object is qualified based on the target parameter of the target model, including: analyzing the target parameter based on the trained classification model to determine whether the first target object is qualified.

[0087] In this embodiment, in response to the model pre-training instruction, the data of different time periods and / or different scenes is labeled to obtain labeled data, which can be manually labeled. The manual labeling is used to pre-train the classification model before the classification model is put into operation. Only the trained classification model can be deployed online.

[0088] In this embodiment, the classification model is trained by the above-mentioned labeled data before the classification model is put into operation. After the trained classification model is deployed online, it can be used to process real-time data that has not been labeled. Optionally, the above-mentioned target model (group intelligence crowd simulation model) is trained by the data of the point in the recent period (such as the last 10 minutes), and the trained classification model is used to analyze the parameters of the target model to determine whether the first target object is qualified, such as judging whether the first target object has illegal operation.

[0089] As an optional implementation, the target data is spatial structured data, and the method further includes: storing the spatial structured data into a target queue; and training the target model based on the target data, including: determining that the storage duration of the spatial structured data in the target queue reaches a predetermined duration, and then training the target model based on the target data.

[0090] In this embodiment, the target data extracted from the target image can be spatially structured data, which can be called pedestrian flow structured data and may include crowd location structured data. After extracting the aforementioned spatially structured data from the target image, this embodiment can store the spatially structured data in a target queue, which can be a crowd simulation queue. Then, the storage duration of the spatially structured data in the target queue is obtained, and it is determined whether the storage duration is greater than a predetermined duration. If it is determined that the storage duration is greater than the predetermined duration, a target model can be trained based on the target data.

[0091] Optionally, the target data in this embodiment is the result of extracting people from images, which is then converted into spatially structured data and stored in a crowd simulation queue as input for training the step target model.

[0092] Optionally, when visually identifying commercial objects, it is difficult to distinguish them from visually similar non-commercial objects such as abandoned pots and pans, tables and tables, and uncovered trucks or tricycles stranded on the roadside due to traffic congestion. Therefore, this application, combined with crowd simulation model estimation, uses a mixed detection method (a problem that occurs in actual operation and is not the desired result) to identify suspected commercial objects such as abandoned pots and pans, tables and tables, and uncovered trucks or tricycles stranded on the roadside due to traffic congestion, thereby effectively avoiding the influence of such objects.

[0093] This invention also provides another method for object detection using swarm intelligence.

[0094] Figure 3 This is a flowchart of another object detection method for swarm intelligence according to an embodiment of the present invention. Figure 3 As shown, the method may include the following steps:

[0095] Step S302: Respond to the video input command applied to the interactive interface and acquire the target video.

[0096] In the technical solution provided in step S302 of the present invention, the video input command can be triggered by the user to input the target video, thereby the embodiment responds to the video input command and obtains the target video, which can be a video obtained by shooting a target area in the target scene.

[0097] Optionally, this embodiment establishes a regional planar map and displays the target video obtained by the video acquisition device on it. The target video can be a pedestrian flow trajectory map, which can reflect the system's ability to detect and track pedestrian flow and can be used as the basic input for training the target model.

[0098] In step S304, in response to the detection instruction acting on the interactive interface, a detection result is output, wherein the detection result is used to indicate whether the first target object in the target video is qualified, and is determined based on the target parameter of the target model.

[0099] In the technical solution provided in step S304 of the present application, after the target video is obtained in response to the video input instruction acting on the interactive interface, the detection result can be output in response to the detection instruction acting on the interactive interface, wherein the detection result is used to indicate whether the first target object in the target video is qualified, and is determined based on the target parameter of the target model, which is trained by the target data in the target video, and the target data is used to represent the behavior information of the first target object.

[0100] In this embodiment, in response to the detection instruction acting on the interactive interface, the target data can be extracted from the target video, wherein the target video can include the first target object to be recognized, and the first target object can be a large number of entities or characters in the target area. Optionally, the first target detection model can be pre-trained, such as a target detection and tracking model, which can be used to identify the first target object in the target video to obtain the target data, which can include a human flow trajectory.

[0101] After the target data is extracted from the target video, the target data can be used as an input to train the target model, and then the target model can be trained based on the target data. The target model can be a swarm intelligence crowd simulation model, which is used to simulate the motion process of the first target object, so that the embodiment achieves the purpose of modeling the flow direction of the crowd in the target area. The target parameter of the target model can be input into the pre-trained classification model, and the target parameter of the target model can be analyzed by the classification model to determine whether the first target object is qualified, and the detection result can be obtained to analyze the flow rule and motivation of the first target object.

[0102] The above method of this embodiment will be further introduced as follows.

[0103] As an optional implementation, a detection box of a second target object in the target video is displayed on the interactive interface, wherein the confidence of the second target object is greater than a first threshold, and the second target object and the target parameter are used to determine whether the first target object is qualified; and / or a behavior trajectory of the first target object is displayed on the interactive interface.

[0104] In this embodiment, a live picture obtained by monitoring the target region by the video acquisition device can be displayed on the interactive interface, a detection frame of the second target object in the target video can be displayed on the live picture, and then whether the first target object is qualified is determined through the second target object and the target parameter, for example, a detection frame of a suspected operating object captured by the second target detection model is output, and a violation operating target is determined based on the suspected operating object and the parameter of the swarm intelligence crowd simulation model. Optionally, the embodiment can also display a behavior trajectory of the first target object in the target region on the interactive interface, and the behavior trajectory is also a people flow trajectory, which reflects the detection and tracking capability of the algorithm on the target in the target video.

[0105] As an optional implementation, the method further includes: when the detection result indicates that the first target object in the target video is unqualified, displaying target information on the interactive interface, wherein the target information is used to represent a reason for a flow direction change of the first target object.

[0106] In this embodiment, an analysis result of a flow direction rule and a cause of the first target object in the target region can be output, so that whether there is unqualified behavior in the target region can be determined, for example, whether there is a violation operating behavior, which can include “normal” / “existence of violation operating behavior” / “congestion (no object produces a violation operating behavior)” and the like. When there is an object that produces a violation operating behavior, target information used to represent a reason for a flow direction change of the first target object in the target region can be displayed, for example, the embodiment estimates a key point of a crowd flow direction change and displays it on the output video, which reflects the filtering effect of the target model on the detection system used to detect the object, and achieves that the accuracy of illegal operation detection can be improved.

[0107] The embodiment of the application further provides another swarm intelligence object detection method.

[0108] Figure 4 is a flowchart of another swarm intelligence object detection method according to the embodiment of the application. As shown in Figure 4 the method can include the following steps:

[0109] In step S402, the target image is obtained by calling the first interface, wherein the first interface includes the first parameter, and the parameter value of the first parameter is the target image.

[0110] In the technical solution provided in the above step S402 of the application, the first interface can be an interface for data interaction between the server and the client. The client can transmit at least one image into the first interface as a first parameter of the first interface, so as to achieve the purpose of uploading the target image to the server.

[0111] Step S404, target data is extracted from the target image, wherein the target data is used to represent the behavior information of the first target object to be detected.

[0112] Step S406, the target model is trained based on the target data, wherein the target model is used to simulate the motion process of the first target object.

[0113] Step S408, whether the first target object is qualified is determined based on the target parameter of the target model, and a detection result is obtained.

[0114] Step S410, the detection result is output by calling the second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the detection result.

[0115] In the technical solution provided by the above step S410 of the present application, the second interface can be an interface for data interaction between a server and a client. The server can transmit the detection result into the second interface as a parameter of the second interface, so as to realize the purpose of issuing the detection result to the client.

[0116] The embodiment comprehensively analyzes and judges whether there is an unqualified first target object in the target region from two dimensions of the second target object and the motion law change of the target object in the target region by combining the method of detecting the second target object and the method of modeling based on the target model. Wherein, by modeling the target data in the target image, the parameters of the target model are obtained, and the parameters of the target model are analyzed, which provides auxiliary information for determining whether the first target object is qualified, avoids the limitation that it is difficult to judge the behavior of the object in the related art based on the computer vision method to identify whether the object is qualified, and also avoids the limitation that only the track and time length of the suspected object can be identified, and the detection ability of whether the object is truly unqualified is still relatively insufficient, which is easy to cause false positives, thereby solving the technical problem of the accuracy of detecting the object, and achieving the technical effect of improving the accuracy of detecting the object.

[0117] Embodiment 2

[0118] The preferred embodiments of the above method of the embodiment are further introduced below, which are specifically explained by taking illegal operation as an example.

[0119] If the object of illegal operation is street occupation, it will cause dirty, messy and poor surrounding environment, which has become a hot issue in promoting urban management, optimizing urban environment and improving urban image, and has attracted widespread attention from all aspects. The way to govern illegal operation through urban management law enforcement is the main way in current urban management. However, the activities of illegal operation are mostly mobile and have strong event occurrence, which greatly increases the difficulty of urban management law enforcement. This method of regulation by human resources has the disadvantages of low efficiency, large consumption of human resources and difficulty in eradicating the problem. Therefore, cameras, artificial intelligence algorithms and other methods can be used to liberate manpower, improve efficiency, and enable law enforcement personnel to focus more on specific scenarios to create a good public environment and order.

[0120] The target detection algorithm can identify the object suspected to be used for operation, the trajectory and length of personnel flow, but this method has poor detection ability for whether there is operation behavior, and is prone to false positives. For example, in the related art, at least one monitoring picture captured by a camera can be obtained, the monitoring picture contains an image of at least one monitoring store, and then store template data corresponding to the monitoring store is obtained, the store template data at least includes a template image of the monitoring store and a global monitoring area thereof, the global monitoring area of the monitoring store in the monitoring picture is obtained according to the template image and the global monitoring area, whether the monitoring store has a violation operation behavior is judged according to the image of the global monitoring area in the monitoring picture, and a first judgment result of the monitoring store is output. This method only uses computer vision methods for template matching and classification based on neural network models, and it is difficult to identify the specific role of the target object and whether there is operation behavior.

[0121] In another related art, monitoring information of a monitoring scene can be obtained from a monitoring terminal every first time interval; the monitoring information is intelligently identified based on a preset first identification model to determine whether there is a violation behavior; if there is, a violation reminder is generated based on the violation behavior; the violation reminder is sent to a patrol terminal at a location information to enable the patrol terminal to send a reminder to a violation personnel; whether the number of violation reminders generated in a second time interval exceeds a preset threshold is determined, wherein the second time interval is greater than the first time interval; if it exceeds, the violation reminder is sent to a law enforcement department; if it does not exceed the preset threshold, the location information and / or the violation behavior are saved into a file. This method can identify city violation behaviors such as facade extension operation, unlicensed operation and street drying occupation by changing the training set through an artificial intelligence (AI) identification model. However, this method ignores the corroborative effect of changes in the flow of surrounding people on the illegal operation behavior, and it is difficult to determine the behavior of the target through identification or classification models, such as misidentifying a non-closed truck in a traffic jam scene as a non-legal operation activity using a motor vehicle, thereby causing false positives.

[0122] When illegal business behavior exists in the scene, the pedestrian flow pattern in the region will change obviously, such as gathering or detouring, etc. Therefore, the application can model the crowd flow direction in the region by combining the swarm intelligence method, analyze the crowd flow direction regularity and cause, and thus can determine whether illegal business behavior exists in the region. The application can further combine the traditional business object detection-based method to improve the accuracy of illegal business detection.

[0123] The above method of the embodiment is further introduced below.

[0124] The embodiment proposes a method for detecting illegal business (such as stall vendors and shop-out business) based on crowd simulation. The method can structure the crowd position and trajectory in the region, model the crowd flow direction based on the swarm intelligence crowd simulation model, learn the change of the parameters in the swarm intelligence crowd simulation model when there is / without business behavior in the region, detect whether there is business behavior in the region according to the path evolution regularity change of the crowd in the region and the identification of suspected business objects by the camera, and thus avoid the defect that the traditional detection method is difficult to identify business behavior.

[0125] Figure 5 is a schematic diagram of an illegal business detection output result according to the embodiment of the application. As shown in Figure 5 , a region plan view is established, on which the crowd trajectory plan view obtained by the camera can be displayed, thereby embodying the ability of the system to detect and track the crowd. The region plan view can be used as the basic input for the swarm intelligence crowd modeling. On the live view of the monitoring camera, the detection frame of the suspected business object captured by the target detection algorithm and the crowd trajectory in the region can be output, thereby embodying the ability of the algorithm to detect and track the target in the image. The embodiment can determine the analysis result of the swarm intelligence model (swarm intelligence crowd simulation model) based on the information displayed on the live view of the monitoring camera, and determine that illegal business behavior exists.

[0126] Optionally, the embodiment can output the analysis result of the cause of the crowd behavior in the region after modeling by the swarm intelligence algorithm, such as outputting the conclusion of "normal", "illegal business behavior", "congestion (no object produces illegal business behavior)", etc. When the object producing illegal business behavior exists, the key point causing the change of the crowd flow direction in the region can be estimated to obtain the target information, and the target information can be displayed on the output image, thereby embodying the filtering effect of the swarm intelligence model on the detection system.

[0127] The embodiment can be based on the assumption that the walking manner of pedestrians changes when illegal business activities exist in the region, and in combination with a target detection algorithm to identify business objects in the picture, comprehensively determine whether there is a violation of business behavior in the detection region, and locate it. The embodiment can enhance the accuracy of the algorithm detection through the behavior information of the surrounding crowd, and reduce the false detection of the target detection algorithm, such as avoiding false detection of abandoned pots, cans, tables, and similar business objects such as roadside vehicles or tricycles due to road congestion.

[0128] Figure 6 is a flowchart of a method for illegal business detection according to an embodiment of the application. As shown in Figure 6 , the method can include the following steps:

[0129] Step S601, reading a target image.

[0130] The embodiment can input a target image, which can include pedestrians.

[0131] Step S602, extracting and storing people flow structured data from the target image.

[0132] The embodiment can extract people flow structured data (spatial structured data) from the target image in combination with the detected historical data, which can be people flow position structured data, which can be stored in the crowd simulation queue. Among them, the position and trajectory of the crowd in the target image can be structured to obtain the people flow structured data.

[0133] Optionally, the embodiment can identify the people in the target image through a pre-trained detection model and tracking model, and can convert the results of the people in the target image into the above-mentioned people flow structured data according to the known deployment position information of the camera.

[0134] Step S603, determining whether the time of storing the people flow data in the crowd simulation queue reaches the target time length.

[0135] If it is determined that the time of storing the people flow data in the crowd simulation queue reaches the target time length, steps S604 and S605 are executed, otherwise, step S601 is executed.

[0136] Step S604, constructing a crowd intelligent crowd simulation model.

[0137] If the time length of storing the people flow structured data in the crowd simulation queue reaches the target time length, the crowd simulation queue can be used to train the crowd intelligent crowd simulation model in a set manner, and the crowd intelligent crowd simulation model can be used to model the crowd flow direction.

[0138] Step S605: detecting whether there is a suspected operating object in the target image based on the target detection model.

[0139] In this embodiment, the target detection algorithm can be used to obtain the target image of the current frame by the target detection model (operating object detection model), and the suspected operating object with a confidence greater than a certain threshold can be detected in the picture of the target image.

[0140] The above-mentioned target detection model of this embodiment can be an operating object detection model pre-trained using offline data, which can detect categories such as backhose, desktop, box, frame, basket, bucket, and stall cloth.

[0141] Optionally, the above-mentioned steps S604 and S605 of this embodiment can be executed in parallel to save time, or can be executed in series to reduce threads.

[0142] Optionally, the detection of the suspected operating object in the above-mentioned step S605 of this embodiment and the identification of the pedestrian in the target image in step S601 can also be performed by training a model that simultaneously contains the target detection of the suspected operating object and the pedestrian, so as to save the time-consuming of calculation.

[0143] Optionally, the target detection model used in the above-mentioned step S605 of this embodiment can also increase or integrate some categories in addition to the original settings, such as splitting the box, frame, basket, and bucket into two categories according to the square and round shape, or unifying the backhose and desktop into the table surface category, and the like.

[0144] Step S606: analyzing the crowd model parameters based on the classification model.

[0145] In this embodiment, the parameters of the crowd intelligence crowd simulation model constructed in step S604 can be composed into a vector, and the vector can be input into the pre-trained classification model, processed by the classification model, and then output the confidence of the existence of illegal operating behavior in the region (if classified as non-existent, the confidence is 0) and the region where illegal operation may exist.

[0146] Optionally, this embodiment can train the crowd intelligence crowd simulation model by collecting crowd data in different time periods / scenes, and manually annotate whether there is illegal operation in the image (video) corresponding to the crowd intelligence crowd simulation model. The parameters of the annotated crowd intelligence crowd simulation model can be input into the above-mentioned classification model for training, which can return the illegal operation behavior classification (yes / no) and the confidence score, and output the region where illegal operation may exist (the activity position where illegal operation may occur). Optionally, in this embodiment, the manually annotated data can be used for pre-training of the above-mentioned classification model. Only the classification model trained by the manually annotated data can be deployed online to process real-time data that has not been annotated.

[0147] Optionally, the classification model in the above step S606 of this embodiment can only output the confidence of whether there is a violation of business operation in the scene, but not output the binary classification (violation of business operation / no violation of business operation) label.

[0148] Step S607, determine whether there is a violation of business operation.

[0149] In this embodiment, if the confidence of the existence of the violation of business operation in step S606 is greater than the set threshold, and it is determined in step S605 that there is a suspected business object, it is determined that there is a violation of business operation, otherwise, step S601 is performed.

[0150] Optionally, even if the above step S607 of this embodiment does not estimate the area where the violation of business operation may exist through the swarm intelligence crowd simulation model, but only determines according to the confidence of the existence of the object (the object is the object where the violation of business operation exists), it can also reduce the false detection effect, and in combination with the estimation of the area where the violation of business operation may exist through the swarm intelligence crowd simulation model, the accuracy of the violation of business operation detection can be further improved.

[0151] Step S608, filter the suspected business object result.

[0152] This embodiment can filter the area in step S605 using the area where the violation of business operation may exist in step S606 to screen out the violation of business operation target. That is, this embodiment filters out the result of the area where the violation of business operation may exist in step S606 from the suspected business object obtained in step S605, and returns the violation of business operation target.

[0153] Step S609, return the violation of business operation target result for reporting.

[0154] This embodiment combines the target detection method and the crowd simulation method based on swarm intelligence to comprehensively analyze and judge whether there is a violation of business operation in the camera area from two dimensions of suspected business objects and changes in swarm intelligence rules in the area. Among them, through the modeling of the swarm intelligence of the crowd in the area, the behavior rules in different scenes are analyzed to provide auxiliary information for the identification of the violation of business operation, which overcomes the limitation that it is difficult to judge the business operation in the related art based on the computer vision method to identify the violation of business operation. By introducing the analysis of swarm intelligence behavior, the false positive rate of the detection of the violation of business operation can be effectively reduced.

[0155] Further, the embodiment introduces analysis of the group intelligence behavior of pedestrians in the image on the basis of the suspected business object detection based on the target detection / identification method, further analyzes the role of the suspected business object in the scene, enables the algorithm to truly identify the business behavior, thereby filtering and screening the results of the illegal business detection, excluding false positives caused by the accumulation of sundries in the scene, and the stopping of trucks / tricycles due to road congestion and other reasons, and improving the accuracy of the algorithm alarm.

[0156] It should be noted that the embodiment can use the open source deep learning algorithm framework pytorch for algorithm development and transplantation to the C++ framework, and the code can be python and C++ code.

[0157] It should be noted that for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0158] From the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0159] Embodiment 3

[0160] According to the embodiments of the present application, a group intelligence object detection method is also provided. Figure 2 The group intelligence object detection device shown in the figure.

[0161] Figure 7 is a schematic diagram of a group intelligence object detection device according to an embodiment of the present application. As shown in the figure, Figure 7 The group intelligence object detection device 70 can include a first acquisition unit 71, a first extraction unit 72, a first training unit 73, and a first determination unit 74.

[0162] The first acquisition unit 71 is configured to acquire a target image.

[0163] The first extraction unit 72 is configured to extract target data from the target image, wherein the target data is used to represent behavior information of the first target object to be detected.

[0164] The first training unit 73 is configured to train a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object.

[0165] The first determination unit 74 is configured to determine whether the first target object is qualified based on a target parameter of the target model.

[0166] It should be noted that the first acquisition unit 71, the first extraction unit 72, the first training unit 73 and the first determination unit 74 correspond to steps S202 to S208 in Embodiment 1, and the four units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0167] According to the embodiments of the present application, a swarm intelligence object detection device for implementing the swarm intelligence object detection method is also provided. Figure 3 As shown in the figure, the swarm intelligence object detection device 80 can include a second acquisition unit 81 and an output unit 82.

[0168] Figure 8 is a schematic diagram of another swarm intelligence object detection device according to the embodiments of the present application. As shown in the figure, the swarm intelligence object detection device 80 can include a second acquisition unit 81 and an output unit 82. Figure 8

[0169] The second acquisition unit 81 is configured to acquire a target video in response to a video input instruction acting on an interactive interface.

[0170] The output unit 82 is configured to output a detection result in response to a detection instruction acting on the interactive interface, wherein the detection result is used to represent whether the first target object in the target video is qualified, and is determined based on a target parameter of a target model, wherein the target model is trained based on target data in the target video, and the target data is used to represent behavior information of the first target object.

[0171] It should be noted that the second acquisition unit 81 and the output unit 82 correspond to steps S302 to S304 in Embodiment 1, and the two units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0172] According to the embodiments of the present application, a swarm intelligence object detection device for implementing the swarm intelligence object detection method is also provided.​Figure 4 The object detection device of the swarm intelligence shown in the object detection method of the swarm intelligence.

[0173] Figure 9 is a schematic diagram of another object detection device of the swarm intelligence according to an embodiment of the present application. As shown in the figure, Figure 9 The object detection device of the swarm intelligence 90 can include a first calling unit 91, a second extraction unit 92, a second training unit 93, a second determination unit 94 and a second calling unit 95.

[0174] The first calling unit 91 is configured to obtain a target image by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter is the target image.

[0175] The second extraction unit 92 is configured to extract target data from the target image, wherein the target data is used to represent the behavior information of the first target object to be detected.

[0176] The second training unit 93 is configured to train a target model based on the target data, wherein the target model is used to simulate the motion process of the first target object.

[0177] The second determination unit 94 is configured to determine whether the first target object is qualified based on the target parameter of the target model, and obtain a detection result.

[0178] The second calling unit 95 is configured to output the detection result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the detection result.

[0179] It should be noted that the first calling unit 91, the second extraction unit 92, the second training unit 93, the second determination unit 94 and the second calling unit 95 correspond to steps S402 to S410 in Embodiment 1, and the two units have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0180] In the object detection device of the swarm intelligence of this embodiment, the target data extracted from the target image can determine the behavior information of the first target object, and then train the target model using the target data to determine the motion process of the target object, and then infer whether the first target object is qualified based on the target parameter of the target model, which avoids only identifying the track and time length of the suspected object, and the detection ability of whether the object is truly unqualified is still relatively lacking, which is easy to cause false alarm, thereby solving the technical problem of the accuracy of detecting the object, and achieving the technical effect of improving the accuracy of detecting the object.

[0181] Embodiment 4

[0182] Embodiments of the present application can provide a swarm intelligence object detection system, which can include a computer terminal, which can be any one of the computer terminal devices in a computer terminal group. Alternatively, in the embodiments, the computer terminal can be replaced by a mobile terminal or other terminal device.

[0183] Alternatively, in the embodiments, the computer terminal can be located in at least one of the network devices in a computer network.

[0184] In the embodiments, the computer terminal can execute program codes of the following steps in the swarm intelligence object detection method of the application program: obtaining a target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; and determining whether the first target object is qualified based on a target parameter of the target model.

[0185] Alternatively, Figure 10 is a structural block diagram of a computer terminal according to an embodiment of the present application. As shown in Figure 10 the computer terminal A can include one or more (only one is shown in the figure) processors 1002, a memory 1004, and a transmission device 1006.

[0186] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the swarm intelligence object detection method and device in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the swarm intelligence object detection method described above. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal A through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0187] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: obtaining a target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; and determining whether the first target object is qualified based on a target parameter of the target model.

[0188] Optionally, the processor can further execute program codes of the following steps: determining that a second target object exists in the target image, wherein a confidence of the second target object is greater than a first threshold; and determining whether the first target object is qualified based on the second target object and the target parameter.

[0189] Optionally, the processor can further execute program codes of the following steps: determining a first region corresponding to an unqualified behavior in the behavior information based on the target parameter, wherein a probability of the unqualified behavior occurring in the first region is greater than a second threshold, and a region corresponding to the target image comprises the first region; and determining whether the first target object is qualified based on the first region and a second region corresponding to the second target object.

[0190] Optionally, the processor can further execute program codes of the following steps: screening the unqualified first target object from the second region based on the first region.

[0191] Optionally, the processor can further execute program codes of the following steps: determining a confidence of the unqualified behavior in the behavior information based on the target parameter; and determining whether the first target object is qualified based on the first region and the second region when the confidence is greater than a third threshold.

[0192] Optionally, the processor can further execute program codes of the following steps: converting the target parameter into a target vector; and determining the first region based on the target vector.

[0193] Optionally, the processor can further execute program codes of the following steps: obtaining labeled data in response to a model pre-training instruction; training a classification model based on the labeled data; and analyzing the target parameter based on the trained classification model to determine whether the first target object is qualified.

[0194] Optionally, the processor can further execute program codes of the following steps: storing the spatial structured data into a target queue; and training a target model based on the target data when a storage duration of the spatial structured data in the target queue reaches a predetermined duration.

[0195] As an optional example, the processor can call information and an application stored in the memory through the transmission device to execute the following steps: obtaining a target video in response to a video input instruction acting on an interactive interface; and outputting a detection result in response to a detection instruction acting on the interactive interface, wherein the detection result is used to indicate whether a first target object in the target video is qualified, and is determined based on a target parameter of a target model, the target model is trained by target data in the target video, and the target data is used to represent behavior information of the first target object.

[0196] As an optional example, the processor can call the information and the application stored in the memory through the transmission device to perform the following steps: obtaining the target image by calling the first interface, wherein the first interface comprises a first parameter, and a parameter value of the first parameter is the target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; determining whether the first target object is qualified based on a target parameter of the target model to obtain a detection result; and outputting the detection result by calling a second interface, wherein the second interface comprises a second parameter, and a parameter value of the second parameter is the detection result.

[0197] By adopting the embodiment of the present application, a swarm intelligence object detection method is provided. The target image is obtained, the target data is extracted from the target image, wherein the target data is used to represent behavior information of a first target object to be detected, the target model is trained based on the target data, wherein the target model is used to simulate a motion process of the first target object, and whether the first target object is qualified is determined based on a target parameter of the target model. That is, the behavior information of the first target object can be determined by the target data extracted from the target image in this embodiment, and then the target model is trained by using the target data to determine the motion process of the target object, and then whether the first target object is qualified is inferred based on the target parameter of the target model, which avoids only being able to identify the track and the time length of the suspected object, and the detection ability of whether the object is truly unqualified is still relatively insufficient, and false positives are easily caused, thereby solving the technical problem of the accuracy of detecting the object, and achieving the technical effect of improving the accuracy of detecting the object.

[0198] Those skilled in the art can understand that, Figure 10 The structure shown is only schematic, and the computer terminal A can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a mobile Internet device (Mobile Internet Devices, abbreviated as MID), a PAD, and the like. Figure 10 It does not limit the structure of the above computer terminal A. For example, the computer terminal A can further comprise more or less components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 10 For example, the computer terminal A can further comprise more or less components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 10 For example, the computer terminal A can further comprise more or less components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0199] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing the terminal device related hardware through a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0200] Embodiment 5

[0201] The embodiments of the present application also provide a computer readable storage medium. Optionally, in the embodiment, the computer readable storage medium can be used to store the program code executed by the object detection method of the swarm intelligence provided in the embodiment 1.

[0202] Optionally, in the embodiment, the computer readable storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0203] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: obtaining a target image; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; and determining whether the first target object is qualified based on a target parameter of the target model.

[0204] Optionally, the computer readable storage medium is further configured to store program code for performing the following steps: determining that a second target object exists in the target image, wherein a confidence of the second target object is greater than a first threshold; and determining whether the first target object is qualified based on the second target object and the target parameter.

[0205] Optionally, the computer readable storage medium is further configured to store program code for performing the following steps: determining a first area corresponding to an unqualified behavior in the behavior information based on the target parameter, wherein a probability of the unqualified behavior occurring in the first area is greater than a second threshold, and an area corresponding to the target image includes the first area; and determining whether the first target object is qualified based on the first area and a second area corresponding to the second target object.

[0206] Optionally, the computer readable storage medium is further configured to store program code for performing the following steps: screening out the unqualified first target object from the second area based on the first area.

[0207] Optionally, the computer readable storage medium is further configured to store program code for performing the following steps: determining a confidence level of the unqualified behavior in the behavior information based on the target parameter; and determining whether the first target object is qualified based on the first region and the second region when the confidence level is greater than a third threshold.

[0208] Optionally, the computer readable storage medium is further configured to store program code for performing the following steps: converting the target parameter into a target vector; and determining the first region based on the target vector.

[0209] Optionally, the computer readable storage medium is further configured to store program code for performing the following steps: obtaining the labeled data in response to the model pre-training instruction; training the classification model based on the labeled data; and determining whether the first target object is qualified based on the analysis of the target parameter by the trained classification model.

[0210] Optionally, the computer readable storage medium is further configured to store program code for performing the following steps: storing the spatial structured data into a target queue; and training the target model based on the target data when a storage duration of the spatial structured data in the target queue reaches a predetermined duration.

[0211] As an optional example, the computer readable storage medium is configured to store program code for performing the following steps: obtaining the target video in response to a video input instruction acting on the interactive interface; and outputting a detection result in response to a detection instruction acting on the interactive interface, wherein the detection result is used to indicate whether the first target object is qualified, and is determined based on a target parameter of a target model, the target model is trained by target data in the target video, and the target data is used to indicate behavior information of the first target object.

[0212] As an optional example, the computer readable storage medium is configured to store program code for performing the following steps: obtaining the target image by calling a first interface, wherein the first interface includes a first parameter, and a parameter value of the first parameter is the target image; extracting target data from the target image, wherein the target data is used to indicate behavior information of the first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; determining whether the first target object is qualified based on a target parameter of the target model to obtain a detection result; and outputting the detection result by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter is the detection result.

[0213] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0214] In the above-mentioned embodiments of the present application, the description of each embodiment is focused on, and the part not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0215] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0216] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0217] In addition, each functional unit in each embodiment of the present application can be integrated in a processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0218] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0219] The above-mentioned is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be regarded as the protection scope of the present application.

Claims

1. A swarm intelligent object detection method, characterized in that, The method comprises: acquiring a target image, wherein the target image is an image obtained by photographing a target region in a target scene; extracting target data from the target image, wherein the target data is used to represent behavior information of a first target object to be detected; training a target model based on the target data, wherein the target model is used to simulate a motion process of the first target object; determining whether the first target object is qualified based on a target parameter of the target model; wherein the target data is extracted from the target image, comprising: identifying the first target object in the target image by using a first target detection model to obtain the target data, wherein the first target detection model is a target detection and tracking model; The method further comprises: detecting a target image by using a second target detection model to obtain a second target object, wherein the confidence of the second target object is greater than a first threshold, and the confidence of the second target object is determined based on the product of the probability of the existence of the second target object and the unqualified probability of the second target object; determining whether the first target object is qualified based on the target parameter of the target model, comprising: processing the target parameter by using a classification model to obtain a first region corresponding to an unqualified behavior in the behavior information, wherein the target region includes the first region; determining whether the first target object is qualified based on the first region and a second region corresponding to the second target object, wherein the target region includes the second region, and the second region is a region where the second target object is located; wherein the second target object is an operating object, and the method further comprises: integrating or splitting the category of the operating object based on the shape of the operating object or the type of the operating object by using the second target detection model.

2. The method of claim 1, wherein, The probability of the unqualified behavior occurring in the first region is greater than a second threshold, and the region corresponding to the target image includes the first region.

3. The method of claim 2, wherein, Determining whether the first target object is qualified based on the first region and the second region comprises: screening the unqualified first target object from the second region based on the first region.

4. The method of claim 2, further comprising: determining a confidence of an unqualified behavior in the behavior information based on the target parameter; determining whether the first target object is qualified based on the first region and the second region corresponding to the second target object, comprising: determining that the confidence is greater than a third threshold, and then determining whether the first target object is qualified based on the first region and the second region.

5. The method of claim 2, wherein determining a first region corresponding to an unqualified behavior in the behavior information based on the target parameter comprises: converting the target parameter into a target vector; determining the first region based on the target vector.

6. The method of any one of claims 1-5, further comprising: The method further comprises: in response to a model pre-training instruction, obtaining labeled data; and training a classification model based on the labeled data. The method further comprises: determining whether the first target object is qualified based on a target parameter of the target model, including: analyzing the target parameter based on the trained classification model to determine whether the first target object is qualified.

7. The method according to any one of claims 1 to 5, characterized in that, The target data is spatially structured data, The method further comprises: storing the spatially structured data in a target queue. The method further comprises: training a target model based on the target data, including: determining that a storage duration of the spatially structured data in the target queue reaches a predetermined duration, and then training the target model based on the target data.

8. A swarm intelligence based object detection method, characterized in that, The method further comprises: In response to a video input instruction acting on an interactive interface, a target video is obtained, wherein the target video is a video obtained by shooting a target region in a target scene; In response to a detection instruction acting on the interactive interface, a detection result is output, wherein the detection result is used to indicate whether a first target object in the target video is qualified, and is determined based on a target parameter of a target model, the target model is trained based on target data in the target video, the target data is used to represent behavior information of the first target object, the target data is obtained by identifying the first target object in the target video using a first target detection model, the first target detection model is a target detection and tracking model, the target video further includes a second target object, the second target object is obtained by detecting the target video using a second target detection model, whether the first target object is qualified is determined based on a first region corresponding to an unqualified behavior in the behavior information and a second region corresponding to the second target object, the target region includes the first region and the second region, the first region is obtained by processing a target parameter using a classification model, the second region is a region where the second target object is located, wherein a confidence of the second target object is greater than a first threshold, the confidence of the second target object is determined based on a product of a probability of the existence of the second target object and an unqualified probability of the second target object, wherein the second target object is a business object, and the second target detection model is further used to integrate or split a category of the business object based on a shape of the business object or a type of the business object.

9. The method of claim 8, wherein, displaying a detection box of a second target object in the target video on the interactive interface, wherein a confidence of the second target object is greater than a first threshold, wherein the second target object and the target parameter are used to determine whether the first target object is qualified; and / or displaying a behavior trajectory of the first target object on the interactive interface.

10. The method of claim 8, wherein, The method further comprises: When the detection result indicates that the first target object in the target video is unqualified, target information is displayed on the interactive interface, where the target information is used to represent the reason for the flow direction change of the first target object.

11. A swarm intelligence based object detection method, characterized in that, Comprise: A target image is obtained by calling a first interface, where the first interface includes a first parameter, a parameter value of the first parameter is the target image, and the target image is an image obtained by photographing a target region in a target scene; Target data is extracted from the target image, where the target data is used to represent behavior information of a first target object to be detected; A target model is trained based on the target data, where the target model is used to simulate a motion process of the first target object; It is determined whether the first target object is qualified based on a target parameter of the target model, and a detection result is obtained; The detection result is output by calling a second interface, where the second interface includes a second parameter, and a parameter value of the second parameter is the detection result; Wherein, the target data is extracted from the target image, including: using a first target detection model to identify the first target object in the target image to obtain the target data, wherein the first target detection model is a target detection and tracking model; The method further comprises: using a second target detection model to detect the target image to obtain a second target object, wherein the confidence of the second target object is greater than a first threshold, the confidence of the second target object is determined based on the product of the probability of the existence of the second target object and the unqualified probability of the second target object, the second target object is a business object, and the second target detection model is further used to integrate or split the category of the business object based on the shape of the business object or the type of the business object; Determining whether the first target object is qualified based on the target parameter of the target model includes: using a classification model to process the target parameter to obtain a first region corresponding to an unqualified behavior in the behavior information, where the target region includes the first region; and determining whether the first target object is qualified based on the first region and a second region corresponding to the second target object, where the target region includes the second region, and the second region is a region where the second target object is located.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program controls the device where the computer-readable storage medium is located to execute the method of any one of claims 1-11 when the program is run by the processor.

13. A processor, comprising: The processor is configured to run a program, and the program is configured to execute the method of any one of claims 1-11 when the program is run.

Images