A target detection object expansion method based on non-maximum suppression and microservices

Abstract

The present invention relates to a method for expanding target detection objects based on non-maximum suppression and microservices, and belongs to the field of artificial intelligence. The present invention deploys a trained algorithm model in IceGrid as a microservice. When detecting unknown targets, multiple microservices are called, and a non-maximum suppression algorithm is added to the post-processing section for optimization. The recognition results of different target recognition algorithms are integrated to obtain ideal target detection results. This method is used to solve the problem of requiring repeated training of deep learning algorithms when adding new target detection types based on the original algorithm model, greatly improving the scalability and development efficiency of target detection algorithms.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and specifically relates to a target detection object expansion method based on non-maximum suppression and microservices. Background Art

[0002] To meet the demands of diverse scenarios, object detection algorithms typically require multiple iterations of parameter adjustments, model training, and data testing to achieve relatively ideal detection results. However, expanding functionality, such as adding more target types, requires retraining the model and retuning parameters, a tedious and time-consuming process. Summary of the Invention

[0003] (1) Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is how to provide a target detection object expansion method based on non-maximum suppression and microservices to solve the problem of repeated training of the deep learning algorithm when adding new target detection types on the basis of the original algorithm model.

[0005] (2) Technical solution

[0006] In order to solve the above technical problems, the present invention proposes a target detection object expansion method based on non-maximum suppression and microservices, which includes the following steps:

[0007] Step S1: Client requests service

[0008] S11. The client defines the data structure and service interface;

[0009] S12. Read the image data and pass the image input size as a parameter to the service interface;

[0010] S13. The client accesses the registration center to query and obtain the service IP address and port information;

[0011] S14. The client transmits parameters to the server according to the service IP address and port information to call the service;

[0012] Step S2: The server processes the request

[0013] S21. The server defines the parameters of the algorithm model service;

[0014] S22. Register service information with the registration center and monitor client requests;

[0015] S23. Receive the request parameters sent by the client and transmit the request parameters to the algorithm model service for inference calculation;

[0016] S24. After the inference is completed, the server returns the inference result to the client.

[0017] Step S3: Client outputs results

[0018] S31: The client receives the inference results of all services and merges the inference results;

[0019] S32, use NMS algorithm to process similar detection targets;

[0020] S33. Detection targets of different categories are directly merged and the final result is given.

[0021] Furthermore, the deployment process of the method includes:

[0022] Deploy the trained target recognition algorithm model as a service in IceGrid and register the service with the registry. The registry stores the service's IP address and port number and listens for client requests.

[0023] When a client sends a service request to the registry, the registry returns the service's IP address and port number to the client.

[0024] The client calls different services based on the IP address and port of the service. After traversing all algorithm models, it calls the NMS algorithm to obtain the target with the highest credibility among similar targets, merges different targets, and obtains the final ideal result.

[0025] Furthermore, when it is necessary to add the function of identifying other types of targets, the algorithm model for identifying the target is deployed in the form of microservices.

[0026] Furthermore, in step S11, the parameters of the data structure include: target name, left top coordinate, right bottom coordinate and target confidence.

[0027] Furthermore, in step S11, the parameters of the service interface include: algorithm name, algorithm type, target type and image input size.

[0028] Furthermore, in step S12, the image input size includes: image height, image width and number of image channels.

[0029] Furthermore, in step S21, the parameters of the algorithm model service include: input image data and output inference results.

[0030] Furthermore, in step S31, the inference results of different algorithm model services are directly connected together to obtain a merged inference result.

[0031] Furthermore, the inference result includes: type, candidate box coordinates and probability value.

[0032] Furthermore, in step S32, multiple candidate boxes obtained by classifiers in different algorithm model services and the probability values ​​of the candidate boxes belonging to the categories are sorted according to the category classification probabilities obtained by the classifier; the scores of all boxes are sorted, and the highest score and its corresponding box are selected; the remaining boxes are traversed, and if the intersection-over-union (IOU) with the current highest-scoring box is greater than a certain threshold, the candidate box is deleted; and the highest-scoring box is selected from the unprocessed boxes, and the above process is repeated.

[0033] (3) Beneficial effects

[0034] The present invention proposes a target detection object expansion method based on non-maximum suppression and microservices. In view of the complex training process and poor functional scalability of target detection algorithm models, the present invention proposes a target detection object expansion method based on non-maximum suppression and microservices. This method deploys different target detection algorithm models in the form of microservices, uses the NMS algorithm to obtain ideal results, avoids repeated training of deep learning algorithms, greatly improves the scalability and development efficiency of target detection algorithms, and facilitates functional expansion and model deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of IOU calculation;

[0036] Figure 2 Schematic diagram of calling the microservice-based target detection algorithm. DETAILED DESCRIPTION

[0037] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0038] Deep learning algorithms are widely used in target detection, face recognition, abnormal behavior analysis, gesture recognition and other fields due to their strong adaptability and high accuracy. Target detection algorithm is a relatively common artificial intelligence algorithm. The algorithm uses methods such as feature extraction and semantic segmentation to identify and mark targets in pictures or videos, thereby realizing target detection, recognition and classification. The application deployment of the algorithm model includes two parts: training and reasoning. The model training process is tedious, time-consuming and labor-intensive. The present invention proposes a target detection object expansion method based on non-maximum suppression and microservices. The method deploys the trained algorithm model in IceGrid in the form of microservices. When detecting unknown targets, multiple microservices are called, and non-maximum suppression algorithms are added to the post-processing part for optimization. The recognition results of different target recognition algorithms are integrated to obtain ideal target detection results. This method is used to solve the problem of repeated training of deep learning algorithms when adding new target detection types on the basis of the original algorithm model, which greatly improves the scalability and development efficiency of the target detection algorithm.

[0039] In view of the complex training process and poor functional scalability of target detection algorithm models, the present invention proposes a target detection object expansion method based on non-maximum suppression and microservices. This method deploys different target detection algorithm models in the form of microservices and uses the NMS algorithm to obtain ideal results, avoiding repeated training of deep learning algorithms, greatly improving the scalability and development efficiency of target detection algorithms, and facilitating functional expansion and model deployment.

[0040] Introduction to NMS algorithm:

[0041] Non-maximum suppression (NMS) is an algorithm that removes non-maxima. It is commonly used in edge detection, object recognition, face detection, and target detection in computer vision. Mainstream target detection algorithms (such as DPM, YOLO, SSD, and FasterR-CNN) all include a process for generating candidate boxes. To ensure a high target detection recall rate, multiple candidate boxes are typically generated. The NMS algorithm is part of the post-processing phase of data processing. Its principle is to search for local maxima and suppress non-maxima. When multiple candidate boxes exist for a target, the best candidate box is selected as the final result.

[0042] Target detection requires locating candidate frames of objects. There is an important concept for the positioning accuracy of candidate frames - Intersection Over Union (IOU). IOU represents the ratio of the intersection area to the union area of ​​two detection frames of the same category. If the IOU is greater than the set threshold, they are considered to be the same target and the candidate frame is discarded. Otherwise, the candidate frame is retained. Figure 1As shown, the calculation formula of the intersection ratio is: select the maximum value of the horizontal and vertical coordinates of the left top corners of the two rectangular boxes, x21, y21; select the minimum value of the horizontal and vertical coordinates of the lower right corners of the two rectangular boxes, x12, y12, then

[0043] The overlapping area inter is:

[0044] inter=|x12-x21|*|y12-y21|

[0045] The union area b is:

[0046] b=|x11-x12|*|y11-y12|+|x21-x22|*|y21-y22|-inter

[0047] The intersection-over-union (IOU) is:

[0048]

[0049] Relying on the classifier in the target detection algorithm to obtain multiple candidate boxes and the probability values ​​of the candidate boxes belonging to the category, the classification probability obtained by the classifier is sorted. The specific algorithm process is as follows:

[0050] (1) Sort the scores of all boxes and select the box with the highest score and its corresponding box;

[0051] (2) Traverse the remaining boxes, and if the intersection-over-union (IOU) with the current highest-scoring box is greater than a certain threshold, delete the candidate box;

[0052] (3) Select the one with the highest score from the unprocessed box and repeat the above process.

[0053] Introduction to Microservices:

[0054] Microservice is a software architecture that uses a modular approach to combine complex applications, and each functional module uses an API that is independent of the programming language to achieve interconnection and intercommunication. The present invention uses the Internet Communication Engine (ICE) middleware to implement microservices. ICE is an object-oriented middleware provided by ZeroC, so that users only need to care about the application itself and do not need to care about how the underlying network communicates. The ICE middleware is defined using an interface description language (IDL) Slice. The main content of the defined communication interface is similar to the Interface type in Java. It is an encapsulation of a series of methods or type declarations and does not involve specific implementation. The Slice compiler provided by ICE compiles the defined Slice file to generate code of other language types. This process maps the ICE service interface to callable code of the corresponding language type (such as C++ / JAVA / python, etc.).

[0055] The ICE client-server framework mainly consists of four parts: the ICE core (ICE Code), proxy, object adapter, and skeleton. The ICE core is responsible for providing underlying support for the normal operation of the client and server; the proxy part replaces the client to implement the function of requesting the server. In essence, the ICE proxy is the local representative or handle of the ICE object; the object adapter mainly completes the mapping from the client request to the corresponding programming language; the skeleton is responsible for providing an upward calling interface for receiving user requests.

[0056] ICEGrid is a powerful service component in ICE that can provide positioning, on-demand activation, application distribution, load balancing and other functions. ICEGrid consists of two parts: a registry and a node. The former is responsible for storing the IP address and port number of the service and monitoring the service process, while the latter manages the relevant information deployed in a specific area and application. Generally, there is only one registry and there can be multiple nodes. Services register with the registry, which stores the IP address and port number of the service and listens for client requests. When the client sends a service request, the registry selects a suitable service address based on the load of the node server and returns it to the client, thereby enabling the client to call the service. The client does not need to know the address and port number of the service, and the server does not need to be fixed to a designated node server. This greatly improves the flexibility and scalability of the server.

[0057] The present invention proposes a target detection object expansion method based on non-maximum suppression and microservices. The method can combine the target detection algorithm models that identify different types, use the NMS algorithm to select the highest score in the same category, delete the remaining detection targets, merge different types of detection targets, and realize the expansion of target detection functions. The client defines the data structure and service interface, reads local or network image data, and passes it to the service interface as a parameter. The client accesses the registration center, queries the number of services and port information, requests the service and transmits the parameters to the server. The server defines the data structure, registers the service information with the registration center and listens to the client request, receives the request parameters sent by the client, and transmits the request parameters to the algorithm model for inference calculation, and returns the result to the client after the inference is completed. After receiving the inference results of all services, the client merges the inference results, uses the NMS algorithm to process the same type of detection targets, directly merges different types of targets, and then gives the final result. The specific steps are as follows.

[0058] Step S1: Client requests service

[0059] S11. The client defines the data structure and service interface;

[0060] The parameters of the data structure Bbox include: target name, left top coordinate, right bottom coordinate, and target confidence; the parameters of the service interface include: algorithm name, algorithm type, target type, and image input size.

[0061] Table 1 Client-defined functions

[0062]

[0063]

[0064] S12. Read the image data and pass the image input size as a parameter to the service interface;

[0065] Image input dimensions include: image height, image width, and number of image channels.

[0066] Table 2 Service interface functions

[0067]

[0068] S13. The client accesses the registration center to query and obtain the service IP address and port information;

[0069] S14. The client transmits the parameters to the server according to the service IP address and port information to call the service.

[0070] Step S2: The server processes the request

[0071] S21. The server defines the parameters of the algorithm model service;

[0072] The parameters of the algorithm model service include: input image data and output inference results.

[0073] Table 3 Server-side defined functions

[0074]

[0075] S22. Register service information with the registration center and monitor client requests;

[0076] S23. Receive the request parameters sent by the client and transmit the request parameters to the algorithm model service for inference calculation;

[0077] Table 4 Algorithm model services

[0078]

[0079] S24. After the inference is completed, the server returns the inference result to the client.

[0080] The inference results include: algorithm type and inference results

[0081] Table 4 Inference results

[0082]

[0083] Step S3: Client outputs results

[0084] S31: The client receives the inference results of all services and merges the inference results;

[0085] The inference results of different algorithm model services are directly connected together to obtain the merged inference results. The inference results include: type, candidate box coordinates, and probability value.

[0086] For example, the inference result of Algorithm 1 is BboxSeq1 = [person, 10, 50, 20, 70, 0.9; car, 20, 15, 50, 30, 0.8], and the inference result of Algorithm 2 is BboxSeq2 = [person, 12, 54, 25, 78, 0.85; house, 10, 70, 20, 35, 0.8], and the merged result is BboxSeq_c = [person, 10, 50, 20, 70, 0.9; car, 20, 15, 50, 30, 0.8; person, 12, 54, 25, 78, 0.85; house, 10, 70, 20, 35, 0.8].

[0087] S32, use NMS algorithm to process similar detection targets;

[0088] BboxSeq=Nms_Rbox(nms,BboxSeq_c);

[0089] Based on the multiple candidate boxes obtained by the classifiers in different algorithm model services and the probability values ​​of the candidate boxes belonging to the categories, they are sorted according to the category classification probabilities obtained by the classifier. The scores of all boxes are sorted, and the highest-scoring box and its corresponding box are selected. The remaining boxes are traversed, and if the intersection-over-union (IOU) ratio with the current highest-scoring box is greater than a certain threshold, the candidate box is deleted. The next box with the highest score is selected from the unprocessed boxes, and the above process is repeated.

[0090] S33. Detection targets of different categories are directly merged and the final result is given.

[0091] For example, BboxSeq = [person, 10, 50, 20, 70, 0.9; car, 20, 15, 50, 30, 0.8; house, 10, 70, 20, 35, 0.8].

[0092] The deployment process is as follows Figure 2 As shown:

[0093] Deploy the trained target recognition algorithm model as a service in IceGrid and register the service with the registry. The registry stores the service's IP address and port number and listens for client requests.

[0094] When a client sends a service request to the registry, the registry returns the service's IP address and port number to the client.

[0095] The client calls different services based on the IP address and port of the service. After traversing all algorithm models, it calls the NMS algorithm to obtain the target with the highest credibility among similar targets, merges different targets, and obtains the final ideal result.

[0096] When you need to add the ability to identify other types of targets, you only need to deploy the algorithm model for identifying the target in the form of microservices, which greatly improves the scalability of the target detection algorithm.

[0097] In view of the complex training process and poor functional scalability of target detection algorithm models, the present invention proposes a target detection object expansion method based on non-maximum suppression and microservices. This method deploys different target detection algorithm models in the form of microservices and uses the NMS algorithm to obtain ideal results, avoiding repeated training of deep learning algorithms, greatly improving the scalability and development efficiency of target detection algorithms, and facilitating functional expansion and model deployment.

[0098] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A target detection object expansion method based on non-maximum suppression and microservices, characterized in that: The method comprises the following steps: Step S1: Client requests service S11. The client defines the data structure and service interface; S12. Read the image data and pass the image input size as a parameter to the service interface; S13. The client accesses the registration center to query and obtain the service IP address and port information; S14. The client transmits parameters to the server according to the service IP address and port information to call the service; in, The deployment process of this method includes: Deploy the trained target recognition algorithm model as a service in IceGrid and register the service with the registry. The registry stores the service's IP address and port number and listens for client requests. When a client sends a service request to the registry, the registry returns the service's IP address and port number to the client. Step S2: The server processes the request S21. The server defines the parameters of the algorithm model service; S22. Register service information with the registration center and monitor client requests; S23. Receive the request parameters sent by the client and transmit the request parameters to the algorithm model service for inference calculation; S24. After the inference is completed, the server returns the inference result to the client. Step S3: Client outputs results S31: The client receives the inference results of all services and merges the inference results; S32, use NMS algorithm to process similar detection targets; The client calls different services based on the service's IP address and port, traverses all algorithm models, and then calls the NMS algorithm to obtain the target with the highest credibility among similar targets. It then merges different targets to obtain the final ideal result. In step S32, multiple candidate boxes obtained by classifiers in different algorithm model services and the probability values ​​of the candidate boxes belonging to the categories are sorted according to the category classification probabilities obtained by the classifier; the scores of all boxes are sorted, and the highest score and its corresponding box are selected; the remaining boxes are traversed, and if the intersection-over-union (IOU) ratio with the current highest-scoring box is greater than a certain threshold, the candidate box is deleted; and the next box with the highest score is selected from the unprocessed boxes, and the above process is repeated; S33. Detection targets of different categories are directly merged and the final result is given.

2. The target detection object expansion method based on non-maximum suppression and microservices according to claim 1, characterized in that: When you need to add the ability to identify other types of targets, deploy the algorithm model for identifying the target in the form of microservices.

3. The target detection object expansion method based on non-maximum suppression and microservices according to claim 1 or 2, characterized in that: In step S11, the parameters of the data structure include: target name, left top coordinate, right bottom coordinate and target confidence.

4. The target detection object expansion method based on non-maximum suppression and microservices according to claim 3, characterized in that: In step S11, the parameters of the service interface include: algorithm name, algorithm type, target type and image input size.

5. The target detection object expansion method based on non-maximum suppression and microservices according to claim 3, characterized in that: In step S12, the image input size includes: image height, image width and number of image channels.

6. The target detection object expansion method based on non-maximum suppression and microservices according to claim 3, characterized in that: In step S21, the parameters of the algorithm model service include: input image data and output inference results.

7. The target detection object expansion method based on non-maximum suppression and microservices according to claim 3, characterized in that: In step S31, the inference results of different algorithm model services are directly connected together to obtain a merged inference result.

8. The target detection object expansion method based on non-maximum suppression and microservices according to claim 7, characterized in that: The inference results include: type, candidate box coordinates and probability value.

Images