Target detection method, system and device and computer equipment

Abstract

The invention relates to a target detection method, system and device and computer equipment. The method is applied to edge end equipment in a traffic monitoring system, and comprises the following steps: determining an initial target detection result according to a monitoring image of a to-be-monitored area, performing reliability evaluation according to the initial target detection result, determining a reliability evaluation result of the initial target detection result, and when the reliability evaluation result is that reliability passes, determining the edge end equipment in the to-be-monitored area. Determining the initial target detection result as a final detection result; the resource processing quantity of the edge end equipment is smaller than that of a cloud server in the traffic monitoring system. By adopting the method, the reliability and the precision of the finally obtained detection result can be ensured to be relatively high, and target detection can be executed by utilizing the edge-end equipment with a relatively small quantity of processing resources in the traffic monitoring system, so that the resources of the edge-end equipment can be utilized to the maximum extent, and the resource utilization rate of the traffic monitoring system is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a target detection method, system, device and computer equipment. Background Art

[0002] With the development of smart transportation, smart traffic monitoring systems are widely used in the field of transportation. In practical applications, traffic jams, violations of pedestrians and vehicles, and other traffic events often occur on the road. Therefore, it is particularly important to use smart traffic monitoring systems to detect targets (such as traffic flow and violations) in traffic event scenarios.

[0003] In the related art, the intelligent traffic monitoring system includes an image acquisition device and a cloud server deployed in the monitored area. The image acquisition device is used to acquire images of the monitored area; the cloud server is used to perform target detection on the images acquired by the image acquisition device to obtain accurate target detection results.

[0004] However, in the process of obtaining accurate target detection, the related technology suffers from low resource utilization. Summary of the invention

[0005] Based on this, it is necessary to provide a target detection method, system, device and computer equipment to address the above technical problems, which can avoid wasting cloud server resources in the traffic monitoring system in the process of obtaining accurate target detection and improve the resource utilization of the traffic monitoring system.

[0006] In a first aspect, an embodiment of the present application provides a target detection method, which is applied to an edge device in a traffic monitoring system; the method comprises:

[0007] Determine an initial target detection result based on a monitoring image of the area to be monitored;

[0008] Perform reliability assessment based on the initial target detection result to determine the reliability assessment result of the initial target detection result;

[0009] When the reliability assessment result is that the reliability passes, the initial target detection result is determined as the final detection result; wherein, the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system.

[0010] The technical solution in the embodiment of the present application is applied to an edge device in a traffic monitoring system, wherein the edge device determines an initial target detection result based on a monitoring image of the area to be monitored, performs a reliability assessment based on the initial target detection result, determines a reliability assessment result of the initial target detection result, and, when the reliability assessment result is a reliability pass, determines the initial target detection result as the final detection result, wherein the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system; the above method can assess the reliability of the initial target detection result of the monitoring image determined by the edge device, and when the initial target detection result is reliable, directly uses the initial target detection result as the final detection result, i.e., the optimal detection result. This process can not only ensure that the reliability and accuracy of the detection result finally obtained are high, but also can utilize The devices with less processing resources in the traffic monitoring system, namely the edge devices, perform target detection, which avoids wasting the resources of the cloud server as much as possible, so that the resources of the edge devices can be maximized, and the resource utilization of the traffic monitoring system is improved; at the same time, the above method does not need to transmit the monitoring image to the cloud server for processing, and avoids the image transmission process during the target detection process, which can not only ensure the security of the monitoring image, but also reduce the target detection delay, improve the real-time response of the target detection, and save a lot of wireless communication bandwidth resources; in addition, when the reliability assessment result of the initial target detection result is reliability passed, the above method does not need the cloud server to participate in completing the target detection, thereby reducing the computing pressure of the cloud server, and based on the above advantages, the traffic monitoring system can be widely used in this field.

[0011] In one embodiment, determining an initial target detection result according to a monitoring image of the area to be monitored includes:

[0012] The surveillance image is input into the lightweight detection model, target detection is performed on the surveillance image, and the initial target detection result is obtained; the lightweight detection model is obtained after the cloud server performs lightweight processing on the deep detection model.

[0013] The technical solution in the embodiment of the present application inputs the monitoring image into a lightweight detection model, performs target detection on the monitoring image, and obtains an initial target detection result, wherein the lightweight detection model is obtained after the cloud server performs lightweight processing on the deep detection model; the device with less processing resources in the above method can use the lightweight detection model for target detection, which can not only reduce the complexity of target detection, but also speed up the speed and efficiency of target detection, improve the response speed of target detection, and thus improve the real-time and timeliness of target detection. On this basis, it can further significantly improve the efficiency and safety of urban traffic management, and provide strong technical support and guarantee for urban traffic management.

[0014] In one embodiment, performing reliability evaluation according to the initial target detection result to determine the reliability evaluation result of the initial target detection result includes:

[0015] Obtain the confidence of the initial target detection result and the complexity measurement index value of the monitoring image; and obtain the resource status of the cloud server;

[0016] According to the complexity measurement index value, confidence level and resource status, the reliability of the initial target detection result is evaluated to determine the reliability evaluation result of the initial target detection result.

[0017] The technical solution in the embodiment of the present application obtains the confidence of the initial target detection result and the complexity measurement index value of the monitoring image; and obtains the resource status of the cloud server, and performs a reliability assessment on the initial target detection result based on the complexity measurement index value, confidence and resource status, and determines the reliability assessment result of the initial target detection result; the above method can perform a reliability assessment on the initial target detection result determined by the edge device, determine whether the initial target detection result is reliable, so as to improve the reference information for determining a higher-precision detection result during the target detection process.

[0018] In one embodiment, performing reliability evaluation on the initial target detection result according to the complexity measurement index value, the confidence level, and the resource status, and determining the reliability evaluation result of the initial target detection result includes:

[0019] The complexity measurement index value, confidence, resource status and initial target detection result are input into the detection result decision model to obtain the decision mark of the initial target detection result;

[0020] According to the decision identifier, the reliability assessment result of the initial target detection result is determined.

[0021] The technical solution in the embodiment of the present application inputs the complexity measurement index value, confidence, resource status and initial target detection result into the detection result decision model, obtains the decision identifier of the initial target detection result, and determines the reliability evaluation result of the initial target detection result according to the decision identifier; when the initial target detection result is determined, the above method can make a decision on the initial target detection result determined by the edge device to determine the reliability evaluation result of the initial target detection result, on this basis, it can provide reference information for finally obtaining a detection result with higher accuracy in the target detection process, and at the same time, the processing does not require human participation, thereby improving the accuracy of the determined decision result and reliability evaluation result; in addition, the above method executes the detection result decision process after the initial detection, which not only improves the flexibility and efficiency of the traffic monitoring system, but also effectively reduces unnecessary communication and computing costs in the target detection process; furthermore, some detection tasks and decision tasks in the above method are run on the edge device, which can reduce the load and maintenance cost of the cloud server in the traffic monitoring system, thereby improving the stability and reliability of the traffic monitoring system.

[0022] In one embodiment, determining a reliability evaluation result of an initial target detection result according to a decision identifier includes:

[0023] If the decision mark is yes, the reliability assessment result is determined to be reliability passed;

[0024] If the decision mark is no, then the reliability evaluation result is determined to be reliability failure.

[0025] This embodiment can determine that the reliability assessment result is reliability passed when the decision mark is yes, and determine that the reliability assessment result is reliability failed when the decision mark is no; this processing process does not require the participation of complex algorithms, thereby reducing the complexity of determining the reliability assessment results and improving the speed and efficiency of the reliability assessment results.

[0026] In one embodiment, the method further comprises:

[0027] During the target detection process, the detection result decision model is optimized according to a preset period to obtain a new detection result decision model.

[0028] In the process of target detection, the present embodiment optimizes the detection result decision model according to a preset period to obtain a new detection result decision model; the above method can continuously optimize the detection result decision model, improve the decision accuracy of the detection result decision model, so that the latest detection result decision model with higher decision accuracy can be applied to execute decisions during the decision-making process, thereby improving the accuracy of the decision results; at the same time, the above method can dynamically update the detection result decision model to adapt to the dynamic changes of traffic scenes, improve the accuracy, decision efficiency and adaptability of the decision method, and make the decision method reliable and stable.

[0029] In one embodiment, the method further comprises:

[0030] When the reliability assessment result is that the reliability fails, a target detection instruction is sent to the cloud server, instructing the cloud server to perform target detection on the monitoring image to obtain the final detection result.

[0031] The technical solution in the embodiment of the present application can instruct the cloud server to perform target detection to obtain a higher-precision detection result when the reliability assessment result of the initial target detection result is that the reliability fails. This process adopts a cloud-edge (i.e., cloud server and edge device) collaborative mechanism to achieve target detection, which can not only ensure that the accuracy of the final detection result is high, but also maximize the utilization of the resources of the cloud server, thereby improving the resource utilization of the traffic monitoring system; at the same time, the above method can realize target detection both on the cloud server and on the edge device, thereby improving the detection performance and detection efficiency of the traffic monitoring system; in addition, the above method adopts a cloud-edge collaborative system to realize target detection, which can form a redundant mechanism. In the event of a failure of the edge device, the target detection can be completed through the cloud server, thereby improving the stability of the operation of the traffic monitoring system.

[0032] In a second aspect, an embodiment of the present application provides a target detection method, which is applied to a cloud server in a traffic monitoring system; the method comprises:

[0033] When the reliability evaluation result of the initial target detection result is that the reliability fails, a monitoring image of the area to be monitored is obtained in response to a target detection instruction sent by an edge device in the traffic monitoring system; wherein the reliability evaluation result is obtained by the edge device determining the initial target detection result according to the monitoring image of the area to be monitored, and performing reliability evaluation according to the initial target detection result; and the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system;

[0034] Perform target detection on the surveillance image to obtain the final detection result.

[0035] The technical solution in the embodiment of the present application, when the reliability assessment result of the initial target detection result is that the reliability fails, in response to the target detection instruction sent by the edge device in the traffic monitoring system, obtains the monitoring image of the area to be monitored, and performs target detection on the monitoring image to obtain the final detection result, wherein the reliability assessment result is obtained by the edge device determining the initial target detection result based on the monitoring image of the area to be monitored, and performing reliability assessment based on the initial target detection result, and the processing resources of the edge device are less than the processing resources of the cloud server in the traffic monitoring system; the above method can only be used when there is a device with less processing resources in the traffic monitoring system, that is, when the reliability assessment result of the initial target detection result determined by the edge device is that the reliability fails, then the cloud server in the traffic monitoring system can be used for target detection to obtain a detection result with higher accuracy, thereby avoiding blindly performing target detection on the monitoring image in any scene through the cloud server during the target detection process. Detection, there is a problem of wasting cloud server resources, so that the cloud server resources can be maximized on the basis of ensuring the accuracy of the final detection results, thereby improving the resource utilization of the traffic monitoring system; at the same time, the above method performs accurate target detection through a cloud server with a large amount of processing resources, which can improve the speed and efficiency of target detection; in addition, in order to ensure that high-precision detection results are finally obtained, the above method needs to consume a large amount of computing resources. Since the processing resources of the cloud server are greater than the processing resources of the edge device, performing target detection on the cloud server side can not only improve the speed and efficiency of target detection, but also improve the utilization of the cloud server; further, the above method can ensure the accuracy of the detection results. On this basis, corresponding measures can be taken based on the accurate detection results to effectively prevent and reduce the occurrence of traffic accidents, help traffic management departments quickly deal with potential safety hazards, and improve road traffic efficiency and urban traffic safety.

[0036] In one embodiment, the above-mentioned object detection on the surveillance image to obtain the final detection result includes:

[0037] The surveillance image is input into a pre-generated deep detection model, target detection is performed on the surveillance image, and the final detection result is obtained; the number of parameters of the deep detection model is greater than the number of parameters of the lightweight detection model.

[0038] The technical solution in the embodiment of the present application inputs the monitoring image into a pre-generated depth detection model, performs target detection on the monitoring image, and obtains the final detection result; the number of parameters of the depth detection model is greater than the number of parameters of the lightweight detection model; in the above method, the cloud server can perform deep processing during the target detection process through the depth detection model with a larger number of parameters, thereby improving the accuracy of the detection results finally obtained by the cloud server.

[0039] In one embodiment, the method further comprises:

[0040] When a depth detection model is generated, the depth detection model is lightweighted to obtain an initial lightweight detection model; the depth detection model is obtained by training the initial detection model according to the sample image set;

[0041] According to the sample image set, the initial lightweight detection model is trained to generate a lightweight detection model;

[0042] The lightweight detection model is sent to the edge device, instructing the edge device to perform target detection.

[0043] The technical solution in the embodiment of the present application is that when a depth detection model is generated, the depth detection model is lightweight processed to obtain an initial lightweight detection model, the initial lightweight detection model is trained according to a sample image set, a lightweight detection model is generated, and the lightweight detection model is sent to an edge device, instructing the edge device to perform target detection; after the depth detection model is lightweight processed, the above method can train the initial lightweight detection model, so that the parameters of the initial lightweight detection model can be optimized to improve the performance and detection accuracy of the lightweight detection model obtained by the edge device; at the same time, the above method can dynamically update the depth detection model and the lightweight detection model, dynamically change the detection accuracy of the depth detection model and the lightweight detection model to adapt to the dynamic changes of traffic scenes and target behaviors, improve the accuracy, detection efficiency and adaptability of the target detection method, and make the target detection method reliable and stable; in addition, in the above method, a high-performance model is converted into a lightweight model to ensure real-time and efficient operation on edge devices with limited processing resources (i.e., computing resources), thereby reducing resource consumption.

[0044] In one embodiment, the method further comprises:

[0045] When the sample image set is updated, obtaining the updated image set;

[0046] The depth detection model is trained according to the updated image set to generate a new depth detection model;

[0047] The new depth detection model is lightweighted to generate a new lightweight detection model, and the new lightweight detection model is sent to the edge device.

[0048] The technical solution in the embodiment of the present application is, when the sample image set is updated, an updated image set is obtained, the depth detection model is trained according to the updated image set to generate a new depth detection model, and the new depth detection model is lightweight processed to generate a new lightweight detection model, and the new lightweight detection model is sent to the edge device; the above method can optimize the parameters of the lightweight detection model and the depth detection model according to the updated image set to improve the performance and detection accuracy of the target detection model obtained by the edge device and the cloud server, and prepare for the determination of more accurate detection results in subsequent applications; at the same time, the above method can dynamically update the depth detection model and the lightweight detection model to adapt to the dynamic changes of traffic scenes and target behaviors, thereby improving the accuracy, detection efficiency and adaptability of the target detection method, and making the target detection method reliable and stable.

[0049] In a third aspect, an embodiment of the present application provides a target detection system, the system comprising: at least one edge device and a cloud server; wherein:

[0050] Each edge device is used to implement the steps of the method of any embodiment of the first aspect above;

[0051] A cloud server, used to implement the steps of the method of any embodiment in the second aspect above.

[0052] In one of the embodiments, the system further comprises: at least one image acquisition device and a storage server;

[0053] Each image acquisition device is used to acquire monitoring images and sample images of the area to be monitored;

[0054] The storage server is used to annotate the sample images, generate a sample image set, and store the sample image set and the monitoring images.

[0055] In a fourth aspect, an embodiment of the present application provides a target detection device, the device comprising:

[0056] An acquisition module is used to determine an initial target detection result based on a monitoring image of the area to be monitored;

[0057] A reliability evaluation module, used to perform reliability evaluation based on the initial target detection result, and determine the reliability evaluation result of the initial target detection result;

[0058] A determination module is used to determine the initial target detection result as the final detection result when the reliability evaluation result is reliability passed; wherein the processing resource amount of the edge device in the traffic monitoring system is less than the processing resource amount of the cloud server in the traffic monitoring system.

[0059] In a fifth aspect, an embodiment of the present application provides a target detection device, the device comprising:

[0060] The command response module is used to respond to the target detection command sent by the edge device in the traffic monitoring system to obtain the monitoring image of the area to be monitored when the reliability evaluation result of the initial target detection result is that the reliability fails; wherein the reliability evaluation result is obtained by the edge device determining the initial target detection result according to the monitoring image of the area to be monitored and performing reliability evaluation according to the initial target detection result; the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system;

[0061] The target detection module is used to perform target detection on the monitoring image to obtain the final detection result.

[0062] In a sixth aspect, an embodiment of the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method of any one of the embodiments of the first and second aspects above when executing the computer program.

[0063] In the seventh aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method of any embodiment of the first aspect and the second aspect are implemented.

[0064] In an eighth aspect, an embodiment of the present application further provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method of any one of the embodiments of the first and second aspects above.

[0065] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A diagram showing an application environment of a target detection method in an embodiment;

[0067] Figure 2 Schematic diagram of a target detection method in one embodiment;

[0068] Figure 3 is a flow chart of a target detection method in another embodiment;

[0069] Figure 4 is a flow chart of a target detection method in another embodiment;

[0070] Figure 5 is a flow chart of a target detection method in another embodiment;

[0071] Figure 6 is a flow chart of a target detection method in another embodiment;

[0072] Figure 7 is a flow chart of a target detection method in another embodiment;

[0073] Figure 8 is a structural block diagram of a target detection device in one embodiment;

[0074] Fig. 9 is a structural block diagram of a target detection device in another embodiment;

[0075] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0077] In the field of smart transportation, traffic incidents such as traffic jams, violations of pedestrians and vehicles, etc. often occur on the road. Therefore, it becomes particularly important to use a smart traffic monitoring system to detect targets (such as traffic flow and violations) in traffic event scenarios. In the related art, the smart traffic monitoring system includes an image acquisition device and a cloud server deployed in the monitored area. Among them, the image acquisition device is used to acquire images of the monitored area; the cloud server is used to perform target detection on the images acquired by the image acquisition device to obtain accurate target detection results. However, in the related art, there is a problem of wasting cloud server resources in the process of obtaining accurate target detection, which makes the resource utilization of the traffic monitoring system low. Based on this, the embodiment of the present application provides a target detection method, which can avoid wasting cloud server resources in the traffic monitoring system in the process of obtaining accurate target detection, and improve the resource utilization of the traffic monitoring system.

[0078] The target detection method provided in the embodiment of the present application can be applied to Figure 1The application environment diagram shown. The application environment is a traffic monitoring system, which is a distributed architecture; the traffic monitoring system includes at least one edge device, a cloud server, at least one image acquisition device and a storage server. Among them, each edge device can be a computer device, and the computer device can be a personal computer, desktop computer, laptop computer, tablet computer and other electronic devices; the cloud server and the storage server can be but not limited to an independent server or a server cluster composed of multiple servers; each image acquisition device can be an electronic device with image acquisition function, such as a camera, a camera, a smart phone, a smart bracelet, a smart watch, a tablet computer, etc. Among them, the above-mentioned cloud server can be called a cloud deep learning server. Optionally, each edge device can be connected to at least one image acquisition device for communication, each image acquisition device can be connected to the storage server for communication, the storage server can be connected to the cloud server for communication, and the cloud server can be connected to each edge device for communication; the above-mentioned communication method can be Wi-Fi, mobile network or Bluetooth connection, etc.; wherein, Figure 1 In the connection relationship, both devices can communicate with each other.

[0079] In the embodiment of the present application, each of the above-mentioned edge devices, storage servers and cloud servers can be deployed with domestic components, which can not only reduce the deployment cost of the traffic monitoring system, but also reduce the risk of strong dependence on external products during target detection. At the same time, in the embodiment of the present application, each edge device and at least one image acquisition device can be wired to improve the security of image or video transmission. In practical applications, the number of servers and the number of edge devices in the cloud server of the traffic monitoring system can be flexibly increased or adjusted according to actual needs and the development of urban traffic monitoring to adapt to target detection tasks of different scales and complexities, so that the traffic monitoring system has good scalability and flexibility. The following embodiment of the present application uses any edge device or cloud server in the traffic monitoring system as the execution subject of the target detection method to introduce the specific process of the target detection method.

[0080] like Figure 2 As shown, it is a flow chart of the target detection method provided in the embodiment of the present application, and the method is applied to the edge device in the traffic monitoring system; the method can be implemented by the following steps:

[0081] S101. Determine an initial target detection result according to a monitoring image of the area to be monitored.

[0082] In practical applications, the above method can be applied to any edge device in a traffic monitoring system; the above traffic monitoring system can be understood as a cloud-edge collaborative system, that is, the traffic monitoring system can include both at least one edge device and a cloud server.

[0083] Specifically, the edge device can obtain a monitoring image collected by at least one image acquisition device deployed in the monitored area, and then use a target detection algorithm to perform target detection on the monitoring image to obtain an initial target detection result. Optionally, the target detection algorithm can be implemented by at least one of a sliding window method, an image pyramid method, a feature extraction method, etc.

[0084] It should be noted here that the above-mentioned monitoring images can be images in scenarios such as medical treatment, robot navigation, and communication. In the embodiment of the present application, the monitoring image is an image in a traffic scenario as an example for explanation.

[0085] S102: Perform reliability assessment based on the initial target detection result to determine the reliability assessment result of the initial target detection result.

[0086] In one implementation, the edge device can pre-train an algorithm model, and then input the initial target detection result into the algorithm model for reliability evaluation, and then output the reliability evaluation result of the initial target detection result.

[0087] In another implementation, the edge device may also perform comparison processing, analysis processing and / or matching processing on the initial target detection result to complete the reliability evaluation and obtain a reliability evaluation result of the initial target detection result.

[0088] In the embodiment of the present application, the reliability evaluation result of the initial target detection result may be reliability passed or reliability failed.

[0089] It should be noted here that the reliability assessment result of the initial target detection result is reliability passed, indicating that the accuracy of the initial target detection result obtained by the edge device is high and the initial target detection result is reliable; the reliability assessment result of the initial target detection result is reliability failed, indicating that the accuracy of the initial target detection result obtained by the edge device is low and the initial target detection result is unreliable.

[0090] S103: If the reliability evaluation result is reliability passed, the initial target detection result is determined as the final detection result. The processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system.

[0091] Further, when the reliability assessment result of the initial target detection result is determined to be reliable, the initial target detection result can be determined as the final detection result of the monitoring image. It should be noted here that the processing resource amount of the above-mentioned edge device can be understood as the amount of idle resources available to the edge device when performing target detection; the processing resource amount of the above-mentioned cloud server can be understood as the amount of idle resources available to the cloud server when performing target detection.

[0092] In one embodiment, after executing the step in S102 above, the method may further include: when the reliability assessment result is that the reliability fails, sending a target detection instruction to the cloud server, instructing the cloud server to perform target detection on the monitoring image to obtain a final detection result.

[0093] This embodiment can instruct the cloud server to perform target detection to obtain a higher-precision detection result when the reliability assessment result of the initial target detection result is that the reliability fails. This process adopts a cloud-edge (i.e., cloud server and edge device) collaborative mechanism to achieve target detection, which can not only ensure that the accuracy of the final detection result is high, but also maximize the utilization of the cloud server resources, thereby improving the resource utilization of the traffic monitoring system; at the same time, the above method can achieve target detection both on the cloud server and on the edge device, thereby improving the detection performance and efficiency of the traffic monitoring system; in addition, the above method adopts a cloud-edge collaborative system to achieve target detection, which can form a redundant mechanism. In the event of a failure of the edge device, target detection can be completed through the cloud server, thereby improving the stability of the operation of the traffic monitoring system.

[0094] The technical solution in the embodiment of the present application is applied to an edge device in a traffic monitoring system, wherein the edge device determines an initial target detection result based on a monitoring image of the area to be monitored, performs a reliability assessment based on the initial target detection result, determines a reliability assessment result of the initial target detection result, and, when the reliability assessment result is a reliability pass, determines the initial target detection result as the final detection result, wherein the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system; the above method can assess the reliability of the initial target detection result of the monitoring image determined by the edge device, and when the initial target detection result is reliable, directly uses the initial target detection result as the final detection result, i.e., the optimal detection result. This process can not only ensure that the reliability and accuracy of the detection result finally obtained are high, but also can utilize The devices with less processing resources in the traffic monitoring system, namely the edge devices, perform target detection, which avoids wasting the resources of the cloud server as much as possible, so that the resources of the edge devices can be maximized, and the resource utilization of the traffic monitoring system is improved; at the same time, the above method does not need to transmit the monitoring image to the cloud server for processing, and avoids the image transmission process during the target detection process, which can not only ensure the security of the monitoring image, but also reduce the target detection delay, improve the real-time response of the target detection, and save a lot of wireless communication bandwidth resources; in addition, when the reliability assessment result of the initial target detection result is reliability passed, the above method does not need the cloud server to participate in completing the target detection, thereby reducing the computing pressure of the cloud server, and based on the above advantages, the traffic monitoring system can be widely used in this field.

[0095] The process of determining the initial target detection result based on the monitoring image of the monitored area is described below. In one embodiment, the step in S101 may include: inputting the monitoring image into a lightweight detection model, performing target detection on the monitoring image, and obtaining an initial target detection result. The lightweight detection model is obtained after the cloud server performs lightweight processing on the deep detection model.

[0096] In practical applications, the above-mentioned lightweight detection model is pre-deployed by the edge device and is deployed according to the lightweight detection model sent by the cloud server in the traffic monitoring system. In practical applications, the edge device can input the monitoring image into the lightweight detection model to perform target detection on the monitoring image and then output the initial target detection result.

[0097] Optionally, the above-mentioned lightweight detection model can be composed of at least one combination of a convolutional neural network model, a fully connected neural network model, a recurrent neural network model, a long short-term memory neural network model, a residual neural network model, etc. In practical applications, both the deep detection model and the lightweight detection model can be a multimodal model, which can be used to perform a variety of different tasks, such as monitoring tasks, surveillance tasks, abnormal event recognition tasks, alarm tasks, etc. In the embodiments of the present application, the use of the target detection task is used as an example for explanation.

[0098] Among them, the process of the cloud server generating a lightweight detection model can be described as: the cloud server can obtain a sample image set, and then train the initial detection model according to the sample image set to obtain a depth detection model, and then lightweight the depth detection model to obtain an initial lightweight detection model, and finally train the initial lightweight detection model according to the sample image set to obtain a lightweight detection model.

[0099] Optionally, the above-mentioned sample image set may include multiple sample images collected in different scenarios (such as medical, transportation, robot navigation, communication, etc.). It should be noted here that the number of parameters of the above-mentioned depth detection model is greater than the number of parameters of the lightweight detection model, and the volume of the depth detection model is greater than the volume of the lightweight detection model. Correspondingly, the complexity of the depth detection model can be greater than the complexity of the lightweight detection model, and the storage space occupied by the depth detection model can be greater than the storage space occupied by the lightweight detection model. Naturally, the depth detection model has higher requirements for computing power resources, and the lightweight detection model has lower requirements for computing power resources. Among them, since the processing resources of the edge device are less than the processing resources of the cloud server, therefore, in the embodiment of the present application, the depth detection model with higher complexity is deployed in the cloud server to realize target detection, and the lightweight detection model with lower complexity is deployed in the edge device to realize target detection.

[0100] It should be noted here that the above-mentioned monitoring image may be an image obtained by preprocessing the original image acquired by the image acquisition device, and the preprocessing may include image scaling, cropping, filtering and other processing.

[0101] The technical solution in the embodiment of the present application inputs the monitoring image into a lightweight detection model, performs target detection on the monitoring image, and obtains an initial target detection result, wherein the lightweight detection model is obtained after the cloud server performs lightweight processing on the deep detection model; the device with less processing resources in the above method can use the lightweight detection model for target detection, which can not only reduce the complexity of target detection, but also speed up the speed and efficiency of target detection, improve the response speed of target detection, and thus improve the real-time and timeliness of target detection. On this basis, it can further significantly improve the efficiency and safety of urban traffic management, and provide strong technical support and guarantee for urban traffic management.

[0102] The following is a description of the process of performing reliability evaluation based on the initial target detection result and determining the reliability evaluation result of the initial target detection result. Figure 3 As shown, the steps in S102 above can be implemented in the following manner:

[0103] S112, obtaining the confidence of the initial target detection result and the complexity measurement index value of the monitoring image; and obtaining the resource status of the cloud server.

[0104] The confidence level of the initial target detection result may be obtained after comparison, analysis and / or matching of the initial target detection result.

[0105] Specifically, the edge device can obtain the confidence of the initial target detection result. In one implementation, the confidence of the initial target detection result can be obtained by comparing the target position in the initial target detection result with the expected target position, and the confidence of the initial target detection result can be obtained by comparing the number of targets in the initial target detection result with the expected number of targets. In another implementation, the initial target detection result can be verified, and then the confidence of the initial target detection result can be determined based on the verification result.

[0106] At the same time, the edge device can obtain the number of targets in the monitoring image, and then obtain the complexity measurement index value of the monitoring image according to the number of targets; the complexity measurement index value of the monitoring image can also be determined according to information such as the density of targets, target types, and image clarity in the monitoring image. Optionally, the complexity measurement index value of the monitoring image can be predetermined or obtained in real time.

[0107] Optionally, the resource status of the above-mentioned cloud server can be divided into three levels, namely, idle state, busy state or working state. Among them, the resource status of the cloud server can also be determined based on information such as the computing power, memory, network bandwidth and delay of the cloud server. In addition, in actual applications, the edge device can obtain the total resource amount and the used resource amount of the cloud server, and subtract the total resource amount and the used resource amount of the cloud server to obtain the remaining resource amount of the cloud server. If the remaining resource amount is greater than the first resource amount, the resource state of the cloud server is determined to be idle; if the remaining resource amount is less than the second resource amount, the resource state of the cloud server is determined to be busy; if the remaining resource amount is between the first resource amount and the second resource amount, the resource state of the cloud server is determined to be working. Optionally, the above-mentioned first resource amount can be greater than the second resource amount; the first resource amount and the second resource amount can both be user-defined and determined, or can be determined based on historical experience values.

[0108] In the target detection process, the resource status of the cloud server can change dynamically, and the resource status of the cloud server in S102 can be the resource status at the current moment. In practical applications, the resource status of the cloud server can be represented by an identifier, such as 0 for idle state, 1 for busy state, 2 for working state, etc.

[0109] In an embodiment of the present application, the edge device can obtain the confidence level of the initial target detection result and the complexity measurement index value of the monitoring image after performing target detection using a lightweight detection model.

[0110] S122. Perform reliability evaluation on the initial target detection result according to the complexity measurement index value, confidence level and resource status, and determine the reliability evaluation result of the initial target detection result.

[0111] In one embodiment, the edge device can perform arithmetic operations, comparisons, matching and / or analysis on the confidence of the initial target detection result, the complexity measurement index value of the monitoring image and the resource status of the cloud server to complete the reliability assessment of the initial target detection result, and determine the reliability assessment result of the initial target detection result according to the processing result. Optionally, the arithmetic operation can be implemented by at least one of addition, subtraction, logarithm, exponential, multiplication, division, etc.

[0112] In another embodiment, the edge device can also pre-train an algorithm model, and then input the confidence of the initial target detection result, the complexity measurement index value of the monitoring image, and the resource status of the cloud server into the algorithm model. The algorithm model performs a reliability assessment on the initial target detection result and outputs a reliability assessment result of the initial target detection result.

[0113] In addition, the edge device can also perform reliability assessment on the initial target detection results based on the computing utilization of the cloud server, the network status, the confidence of the initial target detection results, the complexity measurement index value of the monitoring image, and the resource status of the cloud server to determine the reliability assessment result of the initial target detection results.

[0114] The technical solution in the embodiment of the present application obtains the confidence of the initial target detection result and the complexity measurement index value of the monitoring image; and obtains the resource status of the cloud server, and performs a reliability assessment on the initial target detection result based on the complexity measurement index value, confidence and resource status, and determines the reliability assessment result of the initial target detection result; the above method can perform a reliability assessment on the initial target detection result determined by the edge device, determine whether the initial target detection result is reliable, so as to improve the reference information for determining a higher-precision detection result during the target detection process.

[0115] The following is a description of the process of performing reliability evaluation on the initial target detection result based on the complexity measurement index value, confidence level and resource status to determine the reliability evaluation result of the initial target detection result. Figure 4 As shown, the steps in S122 above can be implemented in the following manner:

[0116] S1221. Input the complexity measurement index value, confidence, resource status and initial target detection result into the detection result decision model to obtain the decision mark of the initial target detection result.

[0117] Among them, the above-mentioned detection result decision model can be implemented by at least one of a convolutional neural network model, a residual neural network model, a long short-term memory neural network model, a recurrent neural network model, a radial basis function neural network model, a linear neural network model, etc.

[0118] Specifically, the edge device can input the confidence of the initial target detection result, the complexity measurement index value of the monitoring image, the resource status of the cloud server, and the initial target detection result into the detection result decision model, and decide whether to use the initial target detection result determined by the edge device as the final detection result or the detection result after target detection by the cloud server as the final detection result, and obtain the decision identifier of the initial target detection result.

[0119] Optionally, the detection result decision model can be obtained by pre-training the initial decision model by the edge device according to the sample data set. Optionally, the sample data set can include different target detection results, confidence levels of different target detection results, complexity measurement index values ​​of different sample images, and different resource states of the cloud server.

[0120] In one embodiment, the above method further includes: during the target detection process, optimizing the detection result decision model according to a preset cycle to obtain a new detection result decision model.

[0121] In the process of target detection, the sample data set can be continuously updated according to a preset period. In practical applications, the detection result decision model can be continuously optimized according to the preset period and the new sample data set obtained each time in the preset period, that is, online learning, to obtain a new detection result decision model.

[0122] Furthermore, the detection result decision model can be continuously updated based on the new detection result decision model obtained each time, so that the new detection result decision model can be applied in the next decision.

[0123] In the process of target detection, the present embodiment optimizes the detection result decision model according to a preset period to obtain a new detection result decision model; the above method can continuously optimize the detection result decision model, improve the decision accuracy of the detection result decision model, so that the latest detection result decision model with higher decision accuracy can be applied to execute decisions during the decision-making process, thereby improving the accuracy of the decision results; at the same time, the above method can dynamically update the detection result decision model to adapt to the dynamic changes of traffic scenes, improve the accuracy, decision efficiency and adaptability of the decision method, and make the decision method reliable and stable.

[0124] S1222. Determine the reliability assessment result of the initial target detection result according to the decision identifier.

[0125] In one embodiment, the edge device may input the decision identifier into an algorithm model, and the algorithm model outputs a reliability assessment result of the initial target detection result.

[0126] In another embodiment, the edge device can search for the decision identifier of the initial target detection result in the association relationship between different decision identifiers and different reliability assessment results, and determine the reliability assessment result corresponding to the found decision identifier in the association relationship as the reliability assessment result of the initial target detection result.

[0127] In one embodiment, the step of determining the reliability assessment result of the initial target detection result according to the decision identifier in the above S1222 may include: if the decision identifier is yes, determining the reliability assessment result as reliability passed; if the decision identifier is no, determining the reliability assessment result as reliability failed.

[0128] In practical applications, the above-mentioned decision identifier can be an edge device identifier or a cloud server identifier; or it can be a positive identifier or a negative identifier. Among them, if the decision identifier is an edge device identifier or a positive identifier, it means that the decision identifier is yes. In this case, the initial target detection result determined by the edge device can be used as the final detection result, indicating that the reliability assessment result of the initial target detection result is reliability passed. If the decision identifier is a cloud server identifier or a negative identifier, it means that the decision identifier is no. In this case, the detection result after the cloud server performs target detection can be used as the final detection result, indicating that the reliability assessment result of the initial target detection result is reliability failed.

[0129] This embodiment can determine that the reliability assessment result is reliability passed when the decision mark is yes, and determine that the reliability assessment result is reliability failed when the decision mark is no; this processing process does not require the participation of complex algorithms, thereby reducing the complexity of determining the reliability assessment results and improving the speed and efficiency of the reliability assessment results.

[0130] The technical solution in the embodiment of the present application inputs the complexity measurement index value, confidence, resource status and initial target detection result into the detection result decision model, obtains the decision identifier of the initial target detection result, and determines the reliability evaluation result of the initial target detection result according to the decision identifier; when the initial target detection result is determined, the above method can make a decision on the initial target detection result determined by the edge device to determine the reliability evaluation result of the initial target detection result, on this basis, it can provide reference information for finally obtaining a detection result with higher accuracy in the target detection process, and at the same time, the processing does not require human participation, thereby improving the accuracy of the determined decision result and reliability evaluation result; in addition, the above method executes the detection result decision process after the initial detection, which not only improves the flexibility and efficiency of the traffic monitoring system, but also effectively reduces unnecessary communication and computing costs in the target detection process; furthermore, some detection tasks and decision tasks in the above method are run on the edge device, which can reduce the load and maintenance cost of the cloud server in the traffic monitoring system, thereby improving the stability and reliability of the traffic monitoring system.

[0131] In one embodiment, the present application also provides a target detection method, which is applied to an edge device in a traffic monitoring system; the method includes the following process:

[0132] (1) Input the monitoring image of the monitored area into the lightweight detection model, perform target detection on the monitoring image, and obtain the initial target detection result; the lightweight detection model is obtained after the cloud server performs lightweight processing on the deep detection model;

[0133] (2) Obtain the confidence of the initial target detection result and the complexity measurement index value of the monitoring image; and obtain the resource status of the cloud server;

[0134] (3) Inputting the complexity measurement index value, confidence, resource status and initial target detection result into the detection result decision model to obtain the decision mark of the initial target detection result; wherein, during the target detection process, the detection result decision model is optimized according to a preset period to obtain a new detection result decision model to be applied in the next decision;

[0135] (4) If the decision mark is yes, the reliability evaluation result is determined to be reliability passed; if the decision mark is no, the reliability evaluation result is determined to be reliability failed;

[0136] (5) When the reliability evaluation result is reliability passed, the initial target detection result is determined as the final detection result; wherein the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system;

[0137] (6) When the reliability assessment result is that the reliability fails, a target detection instruction is sent to the cloud server, instructing the cloud server to perform target detection on the monitoring image to obtain the final detection result.

[0138] The execution process of the above (1) to (6) can be specifically referred to the description of the above embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0139] In order to solve the problem of low resource utilization in the process of obtaining accurate target detection in the related technology, the embodiment of the present application also provides a target detection method, which can improve the resource utilization of the traffic monitoring system in the process of obtaining accurate target detection. Figure 1 The specific process of the target detection method is introduced by using the cloud server in.

[0140] like Figure 5 FIG. 1 is a flow chart of a target detection method provided in an embodiment of the present application, which is applied to a cloud server in a traffic monitoring system; the method can be implemented by the following steps:

[0141] S201. When the reliability evaluation result of the initial target detection result is that the reliability fails, in response to the target detection instruction sent by the edge device in the traffic monitoring system, a monitoring image of the area to be monitored is obtained. The reliability evaluation result is obtained by the edge device determining the initial target detection result according to the monitoring image of the area to be monitored and performing reliability evaluation according to the initial target detection result; the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system.

[0142] In practical applications, the edge device in the traffic monitoring system can determine the initial target detection result based on the monitoring image of the monitored area, and perform reliability evaluation based on the initial target detection result to determine the reliability evaluation result of the initial target detection result. If the reliability evaluation result of the initial target detection result is that the reliability fails, the target detection instruction is sent to the cloud server.

[0143] Furthermore, when the reliability assessment result of the initial target detection result is that the reliability fails, the cloud server in the traffic monitoring system can receive and respond to the target detection instructions sent by the edge device in the traffic monitoring system, and obtain monitoring images of the area to be monitored from local, cloud, disk, hard disk and other locations.

[0144] It should be noted here that the reliability assessment result of the initial target detection result is that the reliability fails, which can be understood as the complexity of the monitoring image is high and the accuracy of the initial target detection result of the monitoring image is low.

[0145] S202: Perform target detection on the monitoring image to obtain a final detection result.

[0146] The cloud server may use a target detection algorithm to perform target detection on the monitoring image to obtain a final detection result of the monitoring image. Optionally, the target detection algorithm may be implemented by at least one of a sliding window method, an image pyramid method, a feature extraction method, etc. However, the accuracy of the target detection algorithm for the monitoring image is greater than the accuracy of the target detection algorithm for the monitoring image in S101.

[0147] The technical solution in the embodiment of the present application, when the reliability assessment result of the initial target detection result is that the reliability fails, in response to the target detection instruction sent by the edge device in the traffic monitoring system, obtains the monitoring image of the area to be monitored, and performs target detection on the monitoring image to obtain the final detection result, wherein the reliability assessment result is obtained by the edge device determining the initial target detection result based on the monitoring image of the area to be monitored, and performing reliability assessment based on the initial target detection result, and the processing resources of the edge device are less than the processing resources of the cloud server in the traffic monitoring system; the above method can only be used when there is a device with less processing resources in the traffic monitoring system, that is, when the reliability assessment result of the initial target detection result determined by the edge device is that the reliability fails, then the cloud server in the traffic monitoring system can be used for target detection to obtain a detection result with higher accuracy, thereby avoiding blindly performing target detection on the monitoring image in any scene through the cloud server during the target detection process. Detection, there is a problem of wasting cloud server resources, so that the cloud server resources can be maximized on the basis of ensuring the accuracy of the final detection results, thereby improving the resource utilization of the traffic monitoring system; at the same time, the above method performs accurate target detection through a cloud server with a large amount of processing resources, which can improve the speed and efficiency of target detection; in addition, in order to ensure that high-precision detection results are finally obtained, the above method needs to consume a large amount of computing resources. Since the processing resources of the cloud server are greater than the processing resources of the edge device, performing target detection on the cloud server side can not only improve the speed and efficiency of target detection, but also improve the utilization of the cloud server; further, the above method can ensure the accuracy of the detection results. On this basis, corresponding measures can be taken based on the accurate detection results to effectively prevent and reduce the occurrence of traffic accidents, help traffic management departments quickly deal with potential safety hazards, and improve road traffic efficiency and urban traffic safety.

[0148] In one embodiment, the step of performing target detection on the surveillance image to obtain the final detection result in the above S202 may include: inputting the surveillance image into a pre-generated depth detection model, performing target detection on the surveillance image, and obtaining the final detection result; the number of parameters of the depth detection model is greater than the number of parameters of the lightweight detection model.

[0149] In practical applications, the above-mentioned depth detection model can be a combination of at least one of a convolutional neural network model, a fully connected neural network model, a recurrent neural network model, a long short-term memory neural network model, a residual neural network model, etc.; in an embodiment of the present application, the above-mentioned depth detection model can be composed of a network model of the yolov series.

[0150] It should be noted here that the number of parameters of the above-mentioned deep detection model is greater than the number of parameters of the lightweight detection model, and the volume of the deep detection model is greater than the volume of the lightweight detection model. Correspondingly, the complexity of the deep detection model can be greater than the complexity of the lightweight detection model, and the storage space occupied by the deep detection model can be greater than the storage space occupied by the lightweight detection model. Naturally, the deep detection model has higher requirements for computing power resources, and the lightweight detection model has lower requirements for computing power resources. Among them, since the processing resources of the edge device are less than the processing resources of the cloud server, therefore, in the embodiment of the present application, the deep detection model with higher complexity is deployed in the cloud server to realize target detection, and the lightweight detection model with lower complexity is deployed in the edge device to realize target detection, so that both the deep detection model and the lightweight detection model can run efficiently in an environment with limited resources.

[0151] Specifically, the cloud server can input the monitoring image into a pre-generated deep detection model, perform target detection on the monitoring image, and obtain the final detection result.

[0152] The technical solution in the embodiment of the present application inputs the monitoring image into a pre-generated depth detection model, performs target detection on the monitoring image, and obtains the final detection result; the number of parameters of the depth detection model is greater than the number of parameters of the lightweight detection model; in the above method, the cloud server can perform deep processing during the target detection process through the depth detection model with a larger number of parameters, thereby improving the accuracy of the detection results finally obtained by the cloud server.

[0153] In actual applications, since the processing resources of the edge device are less than those of the cloud server, the cloud server also needs to train a lightweight detection model for the edge device to perform target detection. The generation process of the lightweight detection model is described below. Figure 6 As shown, the above method may further include the following steps:

[0154] S203. When a depth detection model is generated, lightweight processing is performed on the depth detection model to obtain an initial lightweight detection model; the depth detection model is obtained by training the initial detection model according to the sample image set.

[0155] In practical applications, the cloud server can perform lightweight processing on the pre-generated deep detection model to obtain an initial lightweight detection model. Optionally, the lightweight processing can include model pruning, knowledge distillation, and quantization.

[0156] S204: Train the initial lightweight detection model according to the sample image set to generate a lightweight detection model.

[0157] Furthermore, in order to enable the edge device to obtain a higher-performance detection model, the cloud server can train the initial lightweight detection model obtained in the previous step based on the sample image set to generate a lightweight detection model.

[0158] S205: Send the lightweight detection model to the edge device, instructing the edge device to perform target detection.

[0159] Furthermore, the cloud server can send the lightweight detection model to the edge device, instructing the edge device to perform target detection. The lightweight detection model is less complex, and deploying it on the edge device can reduce the computing resource consumption of the edge device in the process of implementing target detection.

[0160] The technical solution in the embodiment of the present application is that when a depth detection model is generated, the depth detection model is lightweight processed to obtain an initial lightweight detection model, the initial lightweight detection model is trained according to a sample image set, a lightweight detection model is generated, and the lightweight detection model is sent to an edge device, instructing the edge device to perform target detection; after the depth detection model is lightweight processed, the above method can train the initial lightweight detection model, so that the parameters of the initial lightweight detection model can be optimized to improve the performance and detection accuracy of the lightweight detection model obtained by the edge device; at the same time, the above method can dynamically update the depth detection model and the lightweight detection model, dynamically change the detection accuracy of the depth detection model and the lightweight detection model to adapt to the dynamic changes of traffic scenes and target behaviors, improve the accuracy, detection efficiency and adaptability of the target detection method, and make the target detection method reliable and stable; in addition, in the above method, a high-performance model is converted into a lightweight model to ensure real-time and efficient operation on edge devices with limited processing resources (i.e., computing resources), thereby reducing resource consumption.

[0161] In one embodiment, during the target detection process, such as Figure 7 As shown, the above method may further include the following steps:

[0162] S206: When the sample image set is updated, obtain an updated image set.

[0163] In the process of target detection, the sample image set can be dynamically updated, that is, the sample image set is incrementally updated to obtain an updated image set. Specifically, the cloud server can obtain the updated image set from a local location, a cloud location, a disk or a hard disk.

[0164] S207: Train the depth detection model according to the updated image set to generate a new depth detection model.

[0165] Specifically, the cloud server can train the depth detection model based on the updated image set obtained in the previous steps to generate a new depth detection model.

[0166] S208: Perform lightweight processing on the new depth detection model to generate a new lightweight detection model, and send the new lightweight detection model to the edge device.

[0167] Furthermore, the new depth detection model can be lightweight processed to obtain an initial new lightweight detection model, and then the initial new lightweight detection model can be trained according to the updated image set to generate a new lightweight detection model. After that, the new lightweight detection model can be sent to the edge device to instruct the edge device to update the lightweight detection model through the new lightweight detection model.

[0168] The technical solution in the embodiment of the present application is, when the sample image set is updated, an updated image set is obtained, the depth detection model is trained according to the updated image set to generate a new depth detection model, and the new depth detection model is lightweight processed to generate a new lightweight detection model, and the new lightweight detection model is sent to the edge device; the above method can optimize the parameters of the lightweight detection model and the depth detection model according to the updated image set to improve the performance and detection accuracy of the target detection model obtained by the edge device and the cloud server, and prepare for the determination of more accurate detection results in subsequent applications; at the same time, the above method can dynamically update the depth detection model and the lightweight detection model to adapt to the dynamic changes of traffic scenes and target behaviors, thereby improving the accuracy, detection efficiency and adaptability of the target detection method, and making the target detection method reliable and stable.

[0169] In one embodiment, the present application also provides a target detection method, which is applied to a cloud server in a traffic monitoring system; the method includes the following process:

[0170] (1) when the reliability evaluation result of the initial target detection result is that the reliability fails, in response to the target detection instruction sent by the edge device in the traffic monitoring system, a monitoring image of the area to be monitored is obtained; wherein the reliability evaluation result is obtained by the edge device determining the initial target detection result according to the monitoring image of the area to be monitored, and performing reliability evaluation according to the initial target detection result; the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system;

[0171] (2) Input the monitoring image into the pre-generated deep detection model, perform target detection on the monitoring image, and obtain the final detection result; the number of parameters of the deep detection model is greater than the number of parameters of the lightweight detection model;

[0172] Furthermore, the above method may further include the following steps:

[0173] (3) When a depth detection model is generated, the depth detection model is lightweighted to obtain an initial lightweight detection model; the depth detection model is obtained by training the initial detection model according to the sample image set;

[0174] (4) Training the initial lightweight detection model based on the sample image set to generate a lightweight detection model;

[0175] (5) Sending the lightweight detection model to the edge device, instructing the edge device to perform target detection;

[0176] In the target detection process, the above method may further include the following steps:

[0177] (6) when the sample image set is updated, obtaining an updated image set;

[0178] (7) Training the depth detection model according to the updated image set to generate a new depth detection model;

[0179] (8) The new depth detection model is lightweighted to generate a new lightweight detection model, and the new lightweight detection model is sent to the edge device.

[0180] The execution process of the above (1) to (8) can be specifically referred to the description of the above embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0181] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0182] Based on the same inventive concept, the embodiment of the present application also provides a target detection system, such as Figure 1 As shown, the system includes: at least one edge device and a cloud server; wherein:

[0183] Each edge device is used to execute Figure 2-Figure 4 The steps of the method of any one of the embodiments;

[0184] Cloud server for execution Figure 5-Figure 7 The steps of the method of any embodiment.

[0185] The target detection system provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0186] In one embodiment, continue to see Figure 1 As shown, the above system may further include: at least one image acquisition device and a storage server;

[0187] Each image acquisition device is used to acquire monitoring images and sample images of the area to be monitored;

[0188] The storage server is used to annotate the sample images, generate a sample image set, and store the sample image set and the monitoring images.

[0189] In practical applications, each image acquisition device is deployed in a different range in the area to be monitored, and is used to acquire monitoring images and sample images of the area to be monitored.

[0190] At the same time, the storage server can store each sample image, and generate a sample image set after annotating each target in each sample image, so that it can be called at any time in subsequent applications. Optionally, the above-mentioned target can be a vehicle, a pedestrian, a building, etc. In an embodiment of the present application, the above-mentioned storage server can also store the final detection result, so that the user can query the final detection result of the target detection based on the query port of the storage server.

[0191] In addition, the storage server may also obtain the video captured by the image capture device, and then call the video decoding unit to decode the video to obtain a video frame, that is, a sample image or a monitoring image.

[0192] The target detection system provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0193] In addition, based on the same inventive concept, the embodiment of the present application also provides a target detection device for implementing the target detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target detection device embodiments provided below can refer to the limitations on the target detection method above, and will not be repeated here.

[0194] In one embodiment, Figure 8This is a schematic diagram of the structure of a target detection device in one embodiment of the present application. The target detection device provided in the embodiment of the present application can be applied to edge devices in a traffic monitoring system. Figure 8 As shown, the target detection device of the embodiment of the present application may include: an acquisition module 11, a reliability evaluation module 12 and a determination module 13, wherein:

[0195] An acquisition module 11 is used to determine an initial target detection result based on a monitoring image of the area to be monitored;

[0196] A reliability evaluation module 12 is used to perform reliability evaluation based on the initial target detection result and determine the reliability evaluation result of the initial target detection result;

[0197] The determination module 13 is used to determine the initial target detection result as the final detection result when the reliability evaluation result is reliability passed; wherein the processing resource amount of the edge device in the traffic monitoring system is less than the processing resource amount of the cloud server in the traffic monitoring system.

[0198] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0199] In one embodiment, the acquisition module 11 includes: a target detection unit, wherein:

[0200] The target detection unit is used to input the monitoring image into the lightweight detection model, perform target detection on the monitoring image, and obtain initial target detection results; the lightweight detection model is obtained after the cloud server performs lightweight processing on the deep detection model.

[0201] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0202] In one embodiment, the reliability evaluation module 12 includes: an acquisition unit and a reliability evaluation unit, wherein:

[0203] An acquisition unit, used to obtain the confidence of the initial target detection result and the complexity measurement index value of the monitoring image; and to obtain the resource status of the cloud server;

[0204] The reliability evaluation unit is used to perform reliability evaluation on the initial target detection result according to the complexity measurement index value, confidence level and resource status, and determine the reliability evaluation result of the initial target detection result.

[0205] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0206] In one embodiment, the reliability evaluation unit includes: an acquisition subunit and a determination subunit, wherein:

[0207] An acquisition subunit is used to input the complexity measurement index value, confidence, resource status and initial target detection result into the detection result decision model to obtain a decision identifier of the initial target detection result;

[0208] The determination subunit is used to determine the reliability evaluation result of the initial target detection result according to the decision identifier.

[0209] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0210] In one embodiment, the determining subunit is specifically configured to:

[0211] If the decision mark is yes, the reliability assessment result is determined to be reliability passed;

[0212] If the decision mark is no, then the reliability evaluation result is determined to be reliability failure.

[0213] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0214] In one embodiment, the target detection device further includes: a model optimization module, wherein:

[0215] During the target detection process, the detection result decision model is optimized according to a preset period to obtain a new detection result decision model.

[0216] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0217] In one embodiment, the target detection device further includes: an instruction sending module, wherein:

[0218] The instruction sending module is used to send a target detection instruction to the cloud server when the reliability assessment result is that the reliability fails, instructing the cloud server to perform target detection on the monitoring image to obtain the final detection result.

[0219] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0220] In another embodiment, Fig. 9 This is a schematic diagram of the structure of a target detection device in one embodiment of the present application. The target detection device provided in the embodiment of the present application can be applied to a cloud server in a traffic monitoring system. Fig. 9 As shown, the target detection device of the embodiment of the present application may include: a command response module 21 and a target detection module 22, wherein:

[0221] The command response module 21 is used to respond to the target detection command sent by the edge device in the traffic monitoring system to obtain the monitoring image of the area to be monitored when the reliability evaluation result of the initial target detection result is that the reliability fails; wherein the reliability evaluation result is obtained by the edge device determining the initial target detection result according to the monitoring image of the area to be monitored and performing reliability evaluation according to the initial target detection result; the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system;

[0222] The target detection module 22 is used to perform target detection on the monitoring image to obtain the final detection result.

[0223] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0224] In one embodiment, the target detection module 22 is specifically used for:

[0225] The surveillance image is input into a pre-generated deep detection model, target detection is performed on the surveillance image, and the final detection result is obtained; the number of parameters of the deep detection model is greater than the number of parameters of the lightweight detection model.

[0226] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0227] In one embodiment, the target detection device further includes: a lightweight processing module, a first training module and a model sending module; wherein:

[0228] A lightweight processing module is used to perform lightweight processing on the depth detection model when a depth detection model is generated, so as to obtain an initial lightweight detection model; the depth detection model is obtained by training the initial detection model according to the sample image set;

[0229] A first training module is used to train an initial lightweight detection model according to a sample image set to generate a lightweight detection model;

[0230] The model sending module is used to send the lightweight detection model to the edge device, instructing the edge device to perform target detection.

[0231] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0232] In one embodiment, the target detection device further includes: an acquisition module, a second training module and a storage module, wherein:

[0233] An acquisition module, used for acquiring an updated image set when the sample image set is updated;

[0234] A second training module is used to train the depth detection model according to the updated image set to generate a new depth detection model;

[0235] The storage module is used to perform lightweight processing on the new depth detection model, generate a new lightweight detection model, and send the new lightweight detection model to the edge device.

[0236] The target detection device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target detection method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0237] For the specific definition of the target detection device, please refer to the definition of the target detection method above, which will not be repeated here. Each module in the above target detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0238] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide processing power. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sample image sets, sample data sets and monitoring images of the area to be monitored. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target detection method is implemented.

[0239] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0240] In one embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the technical solution in the above-mentioned target detection method embodiment of the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0241] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the technical solution of the above-mentioned target detection method of the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0242] In one embodiment, a computer program product is also provided, including a computer program. When the computer program is executed by a processor, the technical solution of the above-mentioned target detection method of the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0243] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0244] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0245] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target detection method, characterized in that: An edge device applied to a traffic monitoring system; the method comprises: Determine an initial target detection result based on a monitoring image of the area to be monitored; Performing a reliability assessment based on the initial target detection result to determine a reliability assessment result of the initial target detection result; When the reliability assessment result is that the reliability passes, the initial target detection result is determined as the final detection result; wherein, the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system.

2. The method according to claim 1, characterized in that The step of determining the initial target detection result according to the monitoring image of the area to be monitored includes: The monitoring image is input into a lightweight detection model, target detection is performed on the monitoring image, and the initial target detection result is obtained; the lightweight detection model is obtained after the cloud server performs lightweight processing on the deep detection model.

3. The method according to claim 1 or 2, characterized in that: The performing reliability assessment according to the initial target detection result to determine the reliability assessment result of the initial target detection result includes: Obtaining the confidence of the initial target detection result and the complexity measurement index value of the monitoring image; and obtaining the resource status of the cloud server; According to the complexity measurement index value, the confidence level and the resource status, a reliability assessment is performed on the initial target detection result to determine a reliability assessment result of the initial target detection result.

4. The method according to claim 3, characterized in that The performing reliability evaluation on the initial target detection result according to the complexity measurement index value, the confidence level and the resource status to determine the reliability evaluation result of the initial target detection result includes: Inputting the complexity measurement index value, the confidence level, the resource status and the initial target detection result into a detection result decision model to obtain a decision identifier of the initial target detection result; A reliability assessment result of the initial target detection result is determined according to the decision identifier.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: When the reliability assessment result is that the reliability fails, a target detection instruction is sent to the cloud server to instruct the cloud server to perform target detection on the monitoring image to obtain the final detection result.

6. A target detection method, characterized in that: A cloud server used in a traffic monitoring system; the method comprises: In the case where the reliability evaluation result of the initial target detection result is that the reliability fails, in response to the target detection instruction sent by the edge device in the traffic monitoring system, a monitoring image of the area to be monitored is obtained; wherein the reliability evaluation result is obtained by the edge device determining the initial target detection result according to the monitoring image of the area to be monitored, and performing reliability evaluation according to the initial target detection result; the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system; Performing target detection on the monitoring image to obtain the final detection result.

7. The method according to claim 6, characterized in that The performing target detection on the monitoring image to obtain the final detection result includes: The monitoring image is input into a pre-generated depth detection model, target detection is performed on the monitoring image, and the final detection result is obtained; the number of parameters of the depth detection model is greater than the number of parameters of the lightweight detection model.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: When the depth detection model is generated, the depth detection model is lightweight processed to obtain an initial lightweight detection model; the depth detection model is obtained by training the initial detection model according to the sample image set; Training the initial lightweight detection model according to the sample image set to generate a lightweight detection model; The lightweight detection model is sent to the edge device, instructing the edge device to perform target detection.

9. A target detection system, characterized in that: The system includes: at least one edge device and a cloud server; wherein: Each of the edge devices is used to perform the steps of any one of the methods of claims 1 to 5; The cloud server is used to execute the steps of any one of the methods described in claims 6-8.

10. A target detection device, characterized in that: The device comprises: An acquisition module is used to determine an initial target detection result based on a monitoring image of the area to be monitored; A reliability evaluation module, used to perform reliability evaluation based on the initial target detection result, and determine a reliability evaluation result of the initial target detection result; A determination module is used to determine the initial target detection result as the final detection result when the reliability evaluation result is reliability passed; wherein the processing resources of the edge device in the traffic monitoring system are less than the processing resources of the cloud server in the traffic monitoring system.

11. A task processing result determination device, characterized in that: The device comprises: The command response module is used to obtain a monitoring image of the area to be monitored in response to a target detection command sent by an edge device in the traffic monitoring system when the reliability evaluation result of the initial target detection result is that the reliability fails; wherein the reliability evaluation result is obtained by the edge device determining the initial target detection result according to the monitoring image of the area to be monitored and performing reliability evaluation according to the initial target detection result; and the processing resource amount of the edge device is less than the processing resource amount of the cloud server in the traffic monitoring system; The target detection module is used to perform target detection on the monitoring image to obtain the final detection result.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images