An object detection method, system, security inspection machine and storage medium

By deploying the target detection model on brain-like chips, the problems of accuracy and inefficiency in traditional X-ray object detection methods are solved, and more efficient item identification and tracking are achieved.

CN118657920BActive Publication Date: 2024-11-19HUNAN SUKE INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411152152.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-19
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

Traditional X-ray object detection methods have problems with detection accuracy and inefficiency, mainly due to inconsistent identification standards.

Method used

Using the object detection method based on the object detection model, the target image and its corresponding image identification are obtained through brain-like chips, the target object and its position information are determined, and the moving trajectory of the object is determined based on the image identification and position information is determined, and the machine learning neural network is used. Training and optimization of detection models are deployed in brain-like chips to improve detection accuracy and efficiency.

Benefits of technology

It improves the accuracy and efficiency of item detection, can more accurately identify and track target items, and enhances the reliability and efficiency of the detection system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657920B_ABST
    Figure CN118657920B_ABST
Patent Text Reader

Abstract

The embodiments of the present application disclose an object detection method, system, security inspection machine and storage medium, which relate to the field of image processing technology. The method is applied to a brain-like chip, including: obtaining a target image and its corresponding image identifier through a brain-like chip, wherein the target image is any one of a plurality of images to be detected, and each image to be detected is assigned an image identifier in the order of acquisition time. Based on the target detection model deployed in the brain-like chip, the target object in the target image and its corresponding object identifier and location information are determined. When each target object in all images to be detected and its corresponding object identifier and location information are determined, the movement trajectory of each target object is determined according to the image identifier, object identifier and location information of all images to be detected. Based on the detection of the brain-like chip, the detection accuracy and efficiency of the object can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an object detection method, system, security inspection machine and storage medium. Background Art

[0002] Object detection technology is an important technology widely used in security, medicine, industry and other fields. For example, traditional X-ray object detection currently relies mainly on manually set rules and algorithms to identify objects. Its identification standards are often not unified, resulting in low detection accuracy and efficiency in traditional X-ray object detection methods. Summary of the invention

[0003] In view of this, one of the purposes of the present application is to provide an object detection method, system, security inspection machine and storage medium, which can improve the accuracy and efficiency of object detection.

[0004] To achieve the above purpose, the technical solution of this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides an object detection method, comprising:

[0006] Acquire a target image and its corresponding image identifier, wherein the target image is any one of a plurality of images to be detected, and each image to be detected is assigned an image identifier in the order of acquisition time;

[0007] Based on the target detection model, the target object in the target image and its corresponding object identification and location information are determined. The target detection model is deployed in the brain-like chip;

[0008] When all target objects in all the images to be detected and their corresponding object identifications and position information are determined, the movement trajectory of each target object is determined according to the image identifications, object identifications and position information of all the images to be detected.

[0009] In a possible implementation, before acquiring the target image and its corresponding image identifier, the method further includes:

[0010] When a training image set and a test image set are obtained, the training images in the training image set are input into a machine learning neural network for training to obtain a detection model;

[0011] Input the test image in the test image set into the detection model to obtain the test result output by the detection model, the test result includes the test prediction box, the test type and the confidence, and both the training image and the test image include the training object and its corresponding object identification and location information;

[0012] When it is detected that the test result meets the preset conditions, the activation function of the detection model is iteratively adjusted, and the detection model is retrained based on the training image until it is detected that the test result does not meet the preset conditions and the iterative adjustment is stopped to obtain the target detection model;

[0013] Deploy the object detection model in a brain-like chip.

[0014] In a possible implementation, deploying the target detection model in a brain-like chip includes:

[0015] When a model file corresponding to the target detection model is obtained, a model graph corresponding to the model file is determined;

[0016] Divide the model graph to obtain multiple sub-model graphs;

[0017] Under the condition that each sub-model graph is processed respectively according to different preset processing strategies, the performance parameters corresponding to the brain-like chip working under different preset processing strategies are obtained;

[0018] Determine the target processing strategy corresponding to each sub-model graph according to all performance parameters to form a target processing strategy set;

[0019] According to all the target processing strategies in the target processing strategy set, a target model file deployed in the brain-like chip is formed to complete the deployment of the target detection model in the brain-like chip.

[0020] In a possible implementation manner, the preset condition includes at least one of the following:

[0021] The intersection-over-union ratio between the test prediction frame and the standard test frame is less than or equal to a first preset threshold;

[0022] The accuracy between the test type and the standard type is less than or equal to a second preset threshold;

[0023] The recall rate between the test type and the standard type is less than or equal to a third preset threshold.

[0024] In a possible implementation, determining a target object in a target image and its corresponding object identification and location information based on a target detection model includes:

[0025] Based on the target detection model, first feature data and second feature data of the target image are obtained, where the first feature data includes color information, and the second feature data includes image size and image identification;

[0026] Determine the target object and its corresponding quantity information and location information according to the first feature data of the target image and the second feature data of the target image;

[0027] Based on the target detection model, a sub-image corresponding to the target object and first feature data and second feature data of the sub-image are obtained;

[0028] The type of the target object is determined according to the first feature data of the sub-image and the second feature data of the sub-image.

[0029] In a possible implementation, before acquiring the first feature data and the second feature data of the target image, the method further includes:

[0030] Perform equal-proportion scaling, padding and normalization processing on the target image to obtain a processed target image;

[0031] Acquiring first feature data and second feature data of a target image, including:

[0032] The first feature data and the second feature data of the processed target image are obtained.

[0033] In a possible implementation, the brain-like chip is further connected to an external alarm. After determining the type of the target object, the method further includes:

[0034] When an alarm instruction is generated, the alarm instruction is sent to the alarm, and the alarm instruction is used to instruct the alarm to work.

[0035] In a second aspect, an embodiment of the present application provides an object detection system, the system comprising:

[0036] An acquisition module is used to acquire a target image and its corresponding image identifier, wherein the target image is any one of a plurality of images to be detected, and each image to be detected is assigned an image identifier in the order of acquisition time;

[0037] A determination module, used to determine the target object in the target image and its corresponding object identification and location information based on the target detection model, where the target detection model is deployed in the brain-like chip;

[0038] The tracking module is used to determine the moving track of each target object according to the image identification, object identification and position information of all the images to be detected, when determining each target object in all the images to be detected and its corresponding object identification and position information.

[0039] In a third aspect, an embodiment of the present application provides a security inspection machine, which includes a memory and a brain-like chip. A computer program is stored in the memory. When the computer program is executed by the brain-like chip, the object detection method provided in the first aspect is implemented.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a brain-like chip, the object detection method provided in the first aspect is implemented.

[0041] An object detection method provided in an embodiment of the present application can obtain a target image and its corresponding image identifier through a brain-like chip, wherein the target image is any one of a plurality of images to be detected, and each image to be detected is assigned an image identifier in the order of acquisition time. Then, based on the target detection model deployed in the brain-like chip, the target object in the target image and its corresponding object identifier and location information are determined. Finally, when each target object in all images to be detected and its corresponding object identifier and location information are determined, the movement trajectory of each target object is determined based on the image identifier, object identifier and location information of all images to be detected. Based on brain-like chip detection, the detection accuracy and efficiency of the object can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. It should be understood that the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0043] Figure 1 A flowchart of an object detection method provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of functional modules of an object detection system provided in an embodiment of the present application;

[0045] Figure 3 This is a diagram of the internal structure of a security inspection machine provided in an embodiment of the present application.

[0046] Description of reference numerals:

[0047] Object detection system 200 , acquisition module 210 , determination module 220 , tracking module 230 . DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0049] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0050] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0051] In various embodiments of the present application, the expression "or" or "at least one of A or / and B" includes any combination or all combinations of the words listed at the same time. For example, the expression "A or B" or "at least one of A or / and B" may include A, may include B, or may include both A and B.

[0052] In the description of the present application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, it is based on the orientation or position relationship shown in the drawings, or is the orientation or position relationship in which the product of the invention is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0053] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0054] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0055] Furthermore, in the embodiments of the present application, the term "connection" may refer to "electrical connection" or "direct connection". "Electrical connection" may refer to direct electrical connection between two components or electrical connection between two components via one or more normally open tubes or other components.

[0056] To facilitate a better understanding of the solutions of the embodiments of the present application, the relevant technologies are first introduced below.

[0057] Brain-like chips are hardware inspired by biological nervous systems and are chips that simulate the way the brain works. This chip simulates the basic functions of the human brain, such as neurons and synapses, and connects these neuromorphic devices into an artificial neural network to simulate the complex functions of the "brain" such as information processing and storage. Unlike the traditional von Neumann architecture, brain-like chips are designed using a neuromorphic architecture.

[0058] Loop splitting is to decompose a large loop into multiple small loops to better utilize cache, control memory access patterns, or pave the way for subsequent parallelization and vectorization operations. For example, a large-scale loop may be split, with the outer loop controlling the number of iterations and the inner loop focusing on the calculation within a single iteration, thereby improving data locality and reducing memory latency.

[0059] Vectorization is the process of converting scalar operations in a loop into vector operations so that multiple data elements can be processed in one instruction cycle. This is particularly effective for the Single Instruction Multiple Data (SIMD) architecture and can significantly improve computing efficiency.

[0060] Parallelization is the process of assigning computing tasks to multiple processors or cores to execute simultaneously to speed up the computing process. For example, a brain-like chip has two cores.

[0061] Loop unrolling is the process of combining multiple iterations in a loop body into a longer loop body, or completely unrolling it into a series of static instructions without loops. This helps reduce loop overhead and improve instruction-level parallelism, especially when the number of loop iterations is small.

[0062] Loop fusion is the process of merging originally independent loops into a larger loop to reduce data movement and storage access between loops and improve cache utilization. Loops with similar or continuous data dependencies can be merged to reduce memory read and write operations, thereby improving efficiency.

[0063] In order to solve the technical problems in the background technology, the embodiments of the present application provide an object detection method, system, security inspection machine and storage medium. The object detection method provided by the embodiments of the present application is first introduced below.

[0064] See also Figure 1 , Figure 1 This is a flowchart of an object detection method provided in an embodiment of the present application. The object detection method can be applied to a brain-like chip, an object detection system, or a security inspection machine equipped with a brain-like chip. The following will be introduced from the perspective of application to a brain-like chip. The object detection method specifically includes the following steps:

[0065] Step 110, obtaining a target image and its corresponding image identifier, wherein the target image is any one of a plurality of images to be detected, and each image to be detected is assigned an image identifier in the order of acquisition time;

[0066] Step 120, determining the target object in the target image and its corresponding object identification and location information based on the target detection model, where the target detection model is deployed in the brain-like chip;

[0067] Step 130, after determining each target object in all the images to be detected and its corresponding object identification and position information, determine the movement trajectory of each target object according to the image identification, object identification and position information of all the images to be detected.

[0068] An object detection method provided in an embodiment of the present application can obtain a target image and its corresponding image identifier through a brain-like chip, wherein the target image is any one of a plurality of images to be detected, and each image to be detected is assigned an image identifier in the order of acquisition time. Then, based on the target detection model deployed in the brain-like chip, the target object in the target image and its corresponding object identifier and location information are determined. Finally, when each target object in all images to be detected and its corresponding object identifier and location information are determined, the movement trajectory of each target object is determined based on the image identifier, object identifier and location information of all images to be detected. Based on brain-like chip detection, the detection accuracy and efficiency of the object can be improved.

[0069] The following will be Figure 1 Each step of the method is described in detail.

[0070] In step 110, the brain-like chip can obtain the target image and its corresponding image identifier. Among them, the security inspection machine generally includes a conveyor belt, through which the items to be inspected can be sent to the X-ray scanning area of ​​the security inspection machine, and image information or video information can be obtained in the X-ray scanning area. The brain-like chip can determine the target image based on the image information or video information obtained in the X-ray scanning area.

[0071] Specifically, the image information or video information in the X-ray scanning area can be collected by a monitoring module such as a camera set inside the security inspection machine, and the brain-like chip can determine the target image from the image information or video information collected by the monitoring module such as the camera.

[0072] The multiple images to be detected may be part or all of the multiple images obtained in the X-ray scanning area, or may be all or part of the images in the video information obtained in the X-ray scanning area.

[0073] Taking the video information obtained in the X-ray scanning area as an example, if the recording specification is 30 frames per second, 30 images can be obtained in the X-ray scanning area in 1 second. All 30 images in 1 second can be used as images to be detected, or several images, such as 5 images, from the 30 images can be taken at equal intervals as images to be detected.

[0074] The image identifier can be used to mark each image to be detected, and the image identifier of each image to be detected is unique and non-repetitive.

[0075] In some embodiments, the brain-like chip in the security inspection machine can be connected to the monitoring module in the security inspection machine to realize information transmission between the brain-like chip and the monitoring module. Exemplarily, the brain-like chip can also control the closing and opening of the monitoring module. For example, when the brain-like chip detects that the conveyor belt of the security inspection machine is started, the brain-like chip can generate an opening instruction and send the opening instruction to the monitoring module. The monitoring module starts and operates normally after receiving the opening instruction. The monitoring module can send the collected image information or video information to the brain-like chip during its normal operation.

[0076] The allocation of image identifiers can be done by the monitoring module according to the order of acquisition time during acquisition, or by the brain-like chip according to the order of acquisition time when multiple images to be detected are acquired.

[0077] In some embodiments, in order to reduce the working pressure of brain-like chips, improve the data processing efficiency of brain-like chips, and thereby improve the efficiency of object detection, the security inspection machine may include a main control chip, which can be connected to the monitoring module and the brain-like chip respectively to realize the transmission of data such as image information or video information.

[0078] Among them, data can be transmitted between the monitoring module and the main control chip, and between the brain-like chip and the main control chip through the PCIe (PCI Express, PCIe) protocol, which can achieve efficient data transmission without compressing the image information or video information collected by the monitoring module. In addition, the original data can be directly transmitted, avoiding the loss of key information that may exist after the image information or video information is compressed, improving the reliability of the target image obtained by the brain-like chip, and thus improving the accuracy and reliability of object detection.

[0079] In some embodiments, when the performance of the brain-like chip is sufficient, a data transmission channel can be directly established between the brain-like chip and the monitoring module, and data can be transmitted using the PCIe protocol to achieve the above-mentioned effects, which will not be repeated here.

[0080] In step 120, the brain-like chip can detect the target object and its location information in the target image through the target detection model. When the target object is detected in the target image, an object identifier can be assigned to each target object, and the object identifiers of different target objects are also unique and non-repetitive.

[0081] It should be noted that the same target item A may exist in multiple images to be detected, such as 5 images to be detected, when it passes through the conveyor belt of the security inspection machine. In this case, the target detection model determines that the position information of the target item A in the 5 images to be detected on the corresponding images may be different, but the item identification of the target item A is the same.

[0082] In step 130, the brain-like chip obtains each target object in all the images to be detected and its corresponding object identification and position information. Considering that the image identification is assigned to the images to be detected in the order of acquisition time, the brain-like chip can first sort the multiple images to be detected according to the image identification. Then, according to the sorted multiple images to be detected, the object identification in each image to be detected, and the position information associated with the object identification, the movement trajectory of each target object is determined.

[0083] Among them, after the brain-like chip determines the target object in any image to be detected and its corresponding object identification and location information, it can establish an association relationship between the corresponding location information and object identification, that is, each object identification in the image to be detected has corresponding location information.

[0084] For example, the brain-like chip obtains 15 images to be detected in the X-ray scanning area within 3 seconds, and the target item A only appears in the 10 images to be detected obtained within the first 2 seconds. Each of the first 10 images to be detected has the item identification of the target item A. By determining the item identification of the target item A and the location information associated with the item identification, the movement trajectory of the target item A within the first 2 seconds can be determined.

[0085] In a possible implementation, before acquiring the target image and its corresponding image identifier, the method further includes:

[0086] When a training image set and a test image set are obtained, the training images in the training image set are input into a machine learning neural network for training to obtain a detection model;

[0087] Input the test image in the test image set into the detection model to obtain the test result output by the detection model, the test result includes the test prediction box, the test type and the confidence, and both the training image and the test image include the training object and its corresponding object identification and location information;

[0088] When it is detected that the test result meets the preset conditions, the activation function of the detection model is iteratively adjusted, and the detection model is retrained based on the training image until it is detected that the test result does not meet the preset conditions and the iterative adjustment is stopped to obtain the target detection model;

[0089] Deploy the object detection model in a brain-like chip.

[0090] In this embodiment, a detection model is obtained by training a machine learning neural network, and a target detection model is obtained by optimizing the detection model, and the target detection model is deployed in a brain-like chip. The brain-like chip deployed with the target detection model can realize the detection of target objects and identify the location information of target objects.

[0091] The machine learning neural network generally includes three parts: a backbone network, a head network and a neck network, and the neck network is an optional network. The machine learning neural network in this embodiment includes two parts: a backbone network and a head network, wherein the backbone network includes a deep learning model structure such as CSP DarkNet, wherein the CSPDarkNet structure includes a Conv module, a C2f module and a SPFF module.

[0092] Among them, the Conv module is a basic module in the backbone network, which can extract features and pass them to the next layer. The Conv module includes a convolution layer, a batch normalization layer and an activation function. The Conv module can input the feature map of the previous layer or the original training image, and the output is a feature map after the convolution operation.

[0093] The C2f module can perform feature fusion in the backbone network. The C2f module can input the features of the previous layer and output the feature map after feature fusion. The C2f module includes a Conv module and a Concat layer for cross-layer connection.

[0094] The SPFF module can realize pooling operations of different scales, and combines the feature maps before and after each pooling to realize multi-scale fusion. While reducing the amount of calculation, it increases the receptive field and retains more spatial information. The SPFF module can input the features of the previous layer and output a low-dimensional feature map after multi-scale feature fusion. In general, the low-dimensional feature map output by the SPFF module not only maintains the spatial information, but also incorporates contextual information of different scales, which is conducive to the detection and positioning of objects.

[0095] The head network includes a Conv module and a C2f module. The head network can input low-dimensional feature maps of different scales obtained in the previous layer. After the training of the machine learning neural network is completed and the aforementioned input is a test image in the test image set, the head network can output a test result including a test prediction box, a test type, and a confidence level.

[0096] The test result meets the preset conditions, which may indicate that the detection accuracy of the detection model obtained by inputting the training images in the training image set into the machine learning neural network for training is low. In this case, the detection accuracy of the detection model can be improved by iteratively adjusting the activation function in the detection model.

[0097] In some embodiments, the training image includes an image identifier, a training object feature, an object identifier of the training object, and location information of the training object in the training image. The training image is input into a machine learning neural network for processing to obtain the training object in the training image, the type of the training object, and the location information of the training object in the training image.

[0098] The machine learning neural network includes a first Conv module, a first C2f module, an SPFF module, a third Conv module and a C2f module, wherein the first Conv module, the first C2f module and the SPFF module belong to the backbone network in the machine learning neural network, and the second Conv module and the second C2f module belong to the head network in the machine learning neural network.

[0099] Specifically, the training image is input into the backbone network of the machine learning neural network, and the first Conv module, the first C2f module and the SPFF module in the backbone network are used for processing to obtain feature maps of different scales. The feature maps of different scales are input into the head network of the machine learning neural network, and the second Conv module and the second C2f module of the head network are used for processing to obtain the prediction box, the item type and the confidence, wherein the prediction box can be used to characterize the position information of the training item in the training image, and the item type can be determined by the item identification. If the number of training times is greater than the preset number, the training can be terminated to obtain the detection model.

[0100] The item identifier of the training item is unique and non-repetitive, but the corresponding training item type can be determined by the item identifier. For example, the letter A represents a type of training item, and two items of the same type can be identified by A1 and A2 respectively.

[0101] In some embodiments, the training image may also include a training object name, which is associated with the training object feature. After being processed by the backbone network and the head network, the training object name may be directly obtained, and the type of the training object may be quickly determined by the training object name. The training object name may be expressed in Chinese, English or other languages.

[0102] The features of the object to be trained include color information such as color parameters.

[0103] In some embodiments, the activation function in the detection model comprises a SiLU activation function.

[0104] In some embodiments, a LIF layer can be added before the Conv module in the backbone network and the head network in the machine learning neural network, and the continuous features can be converted into pulse signals through the LIF layer. A LIF layer can also be added before the C2f module and the SPPF module so that these feature aggregation modules can also be in pulse signals. The activation function can also be replaced by the LIF layer, which can further promote the sparse representation and transmission of information.

[0105] Brain-like chips are good at processing sparse matrix operations. The machine learning neural network with the addition of the LIF layer can further adapt to brain-like chips, thereby improving the energy efficiency of brain-like chips. In addition, the processing of pulse signals is more energy-efficient than the processing of continuous features, and is more suitable for mobile devices and edge computing scenarios.

[0106] In a possible implementation, deploying the target detection model in a brain-like chip includes:

[0107] When a model file corresponding to the target detection model is obtained, a model graph corresponding to the model file is determined;

[0108] Divide the model graph to obtain multiple sub-model graphs;

[0109] Under the condition that each sub-model graph is processed respectively according to different preset processing strategies, the performance parameters corresponding to the brain-like chip working under different preset processing strategies are obtained;

[0110] Determine the target processing strategy corresponding to each sub-model graph according to all performance parameters to form a target processing strategy set;

[0111] According to all the target processing strategies in the target processing strategy set, a target model file deployed in the brain-like chip is formed to complete the deployment of the target detection model in the brain-like chip.

[0112] This embodiment determines the target processing strategy corresponding to each sub-model graph through the performance parameters of the brain-like chip working under different preset processing strategies, and then forms a target model file deployed in the brain-like chip according to the target processing strategies of all sub-model graphs to complete the deployment of the target detection model in the brain-like chip. The brain-like chip deployed with the target detection model can identify the target object in the target image and the location information of the target object in the target image.

[0113] The model graph can represent the structure of the target detection model. Dividing the model graph into multiple sub-model graphs can analyze and optimize each part of the target detection model in a more detailed and targeted manner.

[0114] The preset processing strategy includes at least one operation process and operation parameters corresponding to each operation process.

[0115] In some embodiments, the operation process includes at least one of loop splitting, vectorization, parallelization, loop unrolling, and loop fusion. Among them, loop splitting, loop unrolling, and loop fusion are exemplified as follows:

[0116] The operation parameters corresponding to the cyclic segmentation include the number of segmentations;

[0117] The operation parameters corresponding to the loop unrolling include the number of unrolling times;

[0118] The operation parameters corresponding to loop fusion include the number of fusions.

[0119] Among them, different preset processing strategies may include different operating processes and different operating parameters. Different operating processes and different operating parameters are combined together to form multiple preset processing strategies.

[0120] Each sub-model diagram corresponds to a target processing strategy, and the target processing strategies of different sub-model diagrams can be the same or different.

[0121] Taking the processing of a sub-model graph P as an example, the performance parameters include power consumption. Mn different preset processing strategies are used to process the sub-model graph P respectively, and Mn power consumptions are obtained. Assuming that the i-th power consumption Mi is the lowest among the Mn power consumptions, the corresponding i-th preset processing strategy Mi can be determined as the target processing strategy.

[0122] Determining the target processing strategy can also be understood as finding a highly optimized execution plan for the brain-like chip, which can better adapt to the computing needs of the brain-like chip.

[0123] In some embodiments, the performance parameters corresponding to the brain-like chip working under different preset processing strategies include at least one of memory bandwidth, cache size and power consumption.

[0124] In some embodiments, a target processing policy set including a plurality of target processing policies may be determined by the following steps:

[0125] In some embodiments, before dividing the model graph, the model graph can also be processed by folding constant layers, deleting redundant operators, and performing operator fusion, etc., which can reduce memory usage, reduce the size of the final target model file, facilitate the deployment and storage of the target model file, and thereby improve the efficiency of deploying the target detection model into the brain-like chip.

[0126] S1, define search space: define a search space with adjustable operation process and operation parameters.

[0127] S2, build cost model: construct a cost model based on hardware performance characteristics such as bandwidth, parallel capability of computing units, etc. to predict the performance data corresponding to different preset processing strategies.

[0128] S3, automatic tuning: Use genetic algorithms, random search, gradient descent or other heuristic optimization algorithms to automatically search in the search space in S1 to find the target processing strategy and form a target processing strategy set.

[0129] In the automatic search in S3, you can also set stop conditions such as the number of searches to avoid unlimited searches that affect efficiency.

[0130] In a possible implementation manner, the preset condition includes at least one of the following:

[0131] The intersection-over-union ratio between the test prediction frame and the standard test frame is less than or equal to a first preset threshold;

[0132] The accuracy between the test type and the standard type is less than or equal to a second preset threshold;

[0133] The recall rate between the test type and the standard type is less than or equal to a third preset threshold.

[0134] This embodiment can test the accuracy and reliability of the detection model by setting specific preset conditions, and can improve the accuracy and reliability of the determined target detection model.

[0135] Among them, Intersection over Union (IoU) is the ratio of the intersection area to the union area of ​​the two bounding boxes of the test prediction box and the standard test box, which can be used to evaluate the coverage of the test prediction box to the standard test box.

[0136] The values ​​of IoU, precision and recall are all between 0 and 1.

[0137] Among them, the first preset threshold, the second preset threshold and the third preset threshold can be set according to actual conditions, and this embodiment does not make any specific limitations here.

[0138] In a possible implementation, determining a target object in a target image and its corresponding object identification and location information based on a target detection model includes:

[0139] Based on the target detection model, first feature data and second feature data of the target image are obtained, where the first feature data includes color information, and the second feature data includes image size and image identification;

[0140] Determine the target object and its corresponding quantity information and location information according to the first feature data of the target image and the second feature data of the target image;

[0141] Based on the target detection model, a sub-image corresponding to the target object and first feature data and second feature data of the sub-image are obtained;

[0142] The type of the target object is determined according to the first feature data of the sub-image and the second feature data of the sub-image.

[0143] Specifically, the first feature data in the target image includes color information. If the color information is empty, it means that no object is detected in the target image, that is, the quantity information of the target object is 0 and the position information is empty.

[0144] If the color information is not empty, the target object, the quantity information of the target object, and the position information of the target object in the target image can be determined based on the color information based on the target detection model. Based on the target detection model, according to the image identifier of the target image in the second feature data, the image identifier and the position information of the target object can be associated, and then the target image where each position information is located can be determined.

[0145] The area of ​​the sub-image is smaller than that of the target image. Each sub-image includes at least one target object. By identifying the target object through the sub-image, the accuracy of detecting the type of the target object can be improved.

[0146] It is understandable that when the first recognition is performed based on the first feature data and the second feature data of the target image, the existence of the target object, the number of the target objects and the location information of the target objects in the target image can be determined. When the second recognition is performed based on the first feature data and the second feature data of the sub-image, the type of the target object in the sub-image can be further determined.

[0147] After the type of target item is determined, it is possible to quickly determine which target items are dangerous goods or prohibited items that are not allowed to pass through the security inspection machine. Then, based on the location information determined in the first identification, the location information of at least one image to be inspected that arrives per second is analyzed one by one, thereby achieving real-time tracking of dangerous goods and prohibited items.

[0148] This embodiment can accurately determine the type, quantity information and location information of the target object in the target image by performing two inspections on the target image.

[0149] In a possible implementation, before acquiring the first feature data and the second feature data of the target image, the method further includes:

[0150] Perform equal-proportion scaling, padding and normalization processing on the target image to obtain a processed target image;

[0151] Acquiring first feature data and second feature data of a target image, including:

[0152] The first feature data and the second feature data of the processed target image are obtained.

[0153] Before obtaining the first feature data and the second feature data of the target image, the present embodiment may also pre-process the target image according to actual needs, including scaling, padding, and normalization. Among them, scaling and padding can convert the size of the target image to the input size of the target detection model, so that the target detection model can detect and identify the target image normally. Normalization can unify the feature scales of different images, and also facilitate the processing of the target detection model.

[0154] Specifically, when the size of the input target image is larger than the input size of the target detection model, the target image can be first proportionally reduced, then normalized, and finally padded so that the size of the target image meets the input size of the target detection model.

[0155] When the size of the input target image is smaller than the input size of the target detection model, the target image can be normalized first, then the normalized image can be scaled up proportionally, and finally filled.

[0156] Considering that normalization is to normalize all pixel values ​​of the entire image, the larger the size of the target image, the more time-consuming the normalization process is. This embodiment performs different preprocessing operations according to the size of the target image, so that the target image can be normalized at the minimum size, which can reduce the time consumption of the preprocessing operation and thus improve the detection efficiency of object detection.

[0157] In a possible implementation, the brain-like chip is further connected to an external alarm. After determining the type of the target object, the method further includes:

[0158] When an alarm instruction is generated, the alarm instruction is sent to the alarm, and the alarm instruction is used to instruct the alarm to work.

[0159] This embodiment can generate an alarm instruction when it is detected that the target item is a dangerous item or a prohibited item, so that the alarm device can sound an alarm, and can remind the relevant staff of the security inspection machine to improve the inspection of dangerous items and prohibited items.

[0160] Corresponding to the above method embodiment, the present application embodiment also provides an object detection system, see Figure 2 , Figure 2 A schematic diagram of functional modules of an object detection system provided in an embodiment of the present application, wherein the object detection system 200 includes:

[0161] The acquisition module 210 is used to acquire a target image and its corresponding image identifier, wherein the target image is any one of a plurality of images to be detected, and each image to be detected is assigned an image identifier in the order of acquisition time;

[0162] A determination module 220, for determining a target object in a target image and its corresponding object identification and location information based on a target detection model, wherein the target detection model is deployed in a brain-like chip;

[0163] The tracking module 230 is used to determine the moving track of each target object according to the image identification, object identification and location information of all the images to be detected, when determining each target object in all the images to be detected and its corresponding object identification and location information.

[0164] The brain-like chip provided in the embodiment of the present application can achieve the following Figure 1 The various processes implemented in the method embodiments can achieve similar or identical technical effects, and will not be described again here to avoid repetition.

[0165] In some embodiments, the object detection system 200 further includes a training deployment module, which is used to:

[0166] When a training image set and a test image set are obtained, the training images in the training image set are input into a machine learning neural network for training to obtain a detection model;

[0167] Input the test image in the test image set into the detection model to obtain the test result output by the detection model, the test result includes the test prediction box, the test type and the confidence, and both the training image and the test image include the training object and its corresponding object identification and location information;

[0168] When it is detected that the test result meets the preset conditions, the activation function of the detection model is iteratively adjusted, and the detection model is retrained based on the training image until it is detected that the test result does not meet the preset conditions and the iterative adjustment is stopped to obtain the target detection model;

[0169] Deploy the object detection model in a brain-like chip.

[0170] In some embodiments, the training deployment module is further specifically configured to:

[0171] When a model file corresponding to the target detection model is obtained, a model graph corresponding to the model file is determined;

[0172] Divide the model graph to obtain multiple sub-model graphs;

[0173] Under the condition that each sub-model graph is processed respectively according to different preset processing strategies, the performance parameters corresponding to the brain-like chip working under different preset processing strategies are obtained;

[0174] Determine the target processing strategy corresponding to each sub-model graph according to all performance parameters to form a target processing strategy set;

[0175] According to all the target processing strategies in the target processing strategy set, a target model file deployed in the brain-like chip is formed to complete the deployment of the target detection model in the brain-like chip.

[0176] The preconditions include at least one of the following:

[0177] The intersection-over-union ratio between the test prediction frame and the standard test frame is less than or equal to a first preset threshold;

[0178] The accuracy rate between the test type and the standard type is less than or equal to a second preset threshold;

[0179] The recall rate between the test type and the standard type is less than or equal to a third preset threshold.

[0180] In some embodiments, the determination module 220 is further specifically configured to:

[0181] Based on the target detection model, first feature data and second feature data of the target image are obtained, where the first feature data includes color information, and the second feature data includes image size and image identification;

[0182] Determine the target object and its corresponding quantity information and location information according to the first feature data of the target image and the second feature data of the target image;

[0183] Based on the target detection model, a sub-image corresponding to the target object and first feature data and second feature data of the sub-image are obtained;

[0184] According to the first characteristic data of the sub-image and the second characteristic data of the sub-image, dangerous objects in the target objects and their types and location information are determined.

[0185] In some embodiments, the object detection system 200 further includes a preprocessing module, which is used to:

[0186] Perform equal-proportion scaling, padding and normalization processing on the target image to obtain a processed target image;

[0187] Acquiring first feature data and second feature data of a target image, including:

[0188] The first feature data and the second feature data of the processed target image are obtained.

[0189] In some embodiments, the object detection system 200 further includes an alarm module, which is used to:

[0190] When an alarm instruction is generated, the alarm instruction is sent to the alarm, and the alarm instruction is used to instruct the alarm to work.

[0191] This application also provides a security inspection machine, see Figure 3 , Figure 3An internal structural diagram of a security inspection machine provided in an embodiment of the present application. The security inspection machine includes a brain-like chip, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the security inspection machine stores an operating system and may also store a computer program. When the computer program is executed by the brain-like chip, the brain-like chip may implement the object detection method applied to the security inspection machine in the above embodiment. The internal memory may also store a computer program. When the computer program is executed by the brain-like chip, the brain-like chip may execute the object detection method. Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the security inspection machine to which the scheme of the present application is applied. The specific security inspection machine may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0192] The embodiment of the present application also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the brain-like chip, the object detection method in the method embodiment is implemented.

[0193] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by a brain-like chip to implement the various processes of the embodiment of the above-mentioned object detection method, and can achieve similar or identical technical effects. To avoid repetition, it will not be repeated here.

[0194] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0195] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for detecting an object, characterized in that: Applied to brain-like chips, the method comprises: Acquire a target image and its corresponding image identifier, wherein the target image is any one of a plurality of images to be detected, and each of the images to be detected is assigned an image identifier in the order of acquisition time; Determine the target object in the target image and its corresponding object identification and location information based on the target detection model, wherein the target detection model is deployed in the brain-like chip; When each target object in all the images to be detected and its corresponding object identification and position information are determined, a moving track of each target object is determined according to the image identification, object identification and position information of all the images to be detected; Before acquiring the target image and its corresponding image identifier, the method further includes: In the case of obtaining a model file corresponding to the target detection model, determining a model graph corresponding to the model file; Dividing the model graph to obtain multiple sub-model graphs; Under the condition that each sub-model graph is processed respectively according to different preset processing strategies, the performance parameters corresponding to the brain-like chip working under the different preset processing strategies are obtained; Determine the target processing strategy corresponding to each of the sub-model graphs according to all the performance parameters to form a target processing strategy set; According to all target processing strategies in the target processing strategy set, a target model file to be deployed in the brain-like chip is formed to complete the deployment of the target detection model in the brain-like chip; The preset processing strategy includes at least one operation process of loop splitting, vectorization, parallelization, loop unrolling and loop fusion, and operation parameters corresponding to the at least one operation process, the performance parameters include power consumption, the target processing strategy corresponding to each of the sub-model graphs includes a preset processing strategy, and the target processing strategies corresponding to all the sub-model graphs form the target processing strategy set; Wherein, determining the target object in the target image and its corresponding object identification and location information based on the target detection model includes: Based on the target detection model, first feature data and second feature data of the target image are acquired, wherein the first feature data includes color information, and the second feature data includes image size and image identification; Determine the target object and its corresponding quantity information and location information according to the first feature data of the target image and the second feature data of the target image; Based on the target detection model, acquiring a sub-image corresponding to the target object, and first feature data and second feature data of the sub-image; The type of the target object is determined according to the first feature data of the sub-image and the second feature data of the sub-image.

2. The method according to claim 1, characterized in that The method of obtaining the target detection model includes: When a training image set and a test image set are obtained, the training images in the training image set are input into a machine learning neural network for training to obtain a detection model; Inputting a test image in the test image set into the detection model to obtain a test result output by the detection model, wherein the test result includes a test prediction box, a test type, and a confidence level, and both the training image and the test image include training objects and their corresponding object identification and location information; When it is detected that the test result meets the preset conditions, the activation function of the detection model is iteratively adjusted, and the detection model is retrained based on the training image until it is detected that the test result does not meet the preset conditions, and the iterative adjustment is stopped to obtain the target detection model.

3. The method according to claim 2, characterized in that The preset condition includes at least one of the following: The intersection-over-union ratio between the test prediction frame and the standard test frame is less than or equal to a first preset threshold; The accuracy rate between the test type and the standard type is less than or equal to a second preset threshold; The recall rate between the test type and the standard type is less than or equal to a third preset threshold.

4. The method according to claim 1, characterized in that: Before acquiring the first feature data and the second feature data of the target image, the method further includes: Performing proportional scaling, filling and normalization processing on the target image to obtain a processed target image; The acquiring the first feature data and the second feature data of the target image comprises: The first feature data and the second feature data of the processed target image are acquired.

5. The method according to claim 1, characterized in that The brain-like chip is also externally connected to an alarm. After determining the type of the target object, the method further includes: When an alarm instruction is generated, the alarm instruction is sent to the alarm, and the alarm instruction is used to instruct the alarm to work.

6. An object detection system, characterized in that: The system comprises: An acquisition module, used for acquiring a target image and its corresponding image identifier, wherein the target image is any one of a plurality of images to be detected, and each of the images to be detected is assigned an image identifier in the order of acquisition time; A determination module, used to determine the target object in the target image and its corresponding object identification and location information based on a target detection model, wherein the target detection model is deployed in a brain-like chip; A tracking module, for determining the moving track of each target object according to the image identification, object identification and location information of all the images to be detected, when determining each target object in all the images to be detected and its corresponding object identification and location information; A training deployment module, used to determine a model graph corresponding to the model file when a model file corresponding to the target detection model is obtained; Dividing the model graph to obtain multiple sub-model graphs; Under the condition that each sub-model graph is processed respectively according to different preset processing strategies, the performance parameters corresponding to the brain-like chip working under the different preset processing strategies are obtained; Determine the target processing strategy corresponding to each of the sub-model graphs according to all the performance parameters to form a target processing strategy set; According to all target processing strategies in the target processing strategy set, a target model file to be deployed in the brain-like chip is formed to complete the deployment of the target detection model in the brain-like chip; The preset processing strategy includes at least one operation process of loop splitting, vectorization, parallelization, loop unrolling and loop fusion, and operation parameters corresponding to the at least one operation process, the performance parameters include power consumption, the target processing strategy corresponding to each of the sub-model graphs includes a preset processing strategy, and the target processing strategies corresponding to all the sub-model graphs form the target processing strategy set; The determining module is further specifically used for: Based on the target detection model, first feature data and second feature data of the target image are acquired, wherein the first feature data includes color information, and the second feature data includes image size and image identification; Determine the target object and its corresponding quantity information and location information according to the first feature data of the target image and the second feature data of the target image; Based on the target detection model, acquiring a sub-image corresponding to the target object, and first feature data and second feature data of the sub-image; The type of the target object is determined according to the first feature data of the sub-image and the second feature data of the sub-image.

7. A security inspection machine, characterized in that: The security inspection machine includes a memory and a brain-like chip. The memory stores a computer program. When the computer program is executed by the brain-like chip, it implements the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by one or more brain-like chips, implements the method of any one of claims 1 to 5.