Behavior detection methods, devices, and storage media

Abstract

This application discloses a behavior detection method, device, and storage medium. The behavior detection method includes: extracting features from an image to be detected using a behavior detection network to obtain target image features; performing at least one position correction on each candidate detection box based on the target image features, and determining the updated detection box obtained from the last position correction as the corrected detection box for each candidate detection box; in each position correction, determining the updated detection box for each candidate detection box based on the current position information corresponding to each candidate detection box and the first local image features corresponding to the current position information in the target image features; and determining the behavior detection result of the image to be detected based on the second local image features of the corrected detection boxes for each candidate detection box, wherein the second local image features are the features of the corresponding region of each corrected detection box in the target image features. This scheme can improve the accuracy of behavior detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a behavior detection method, device and storage medium. BACKGROUND

[0002] In an office, it is often necessary to regulate the behavior of staff, such as telephone behavior. In order to improve the supervision efficiency, an intelligent telephone behavior detection system is needed. The current detection algorithm adopts a human preset detection frame for sampling. Since the human preset detection frame is easily affected by the experience of personnel, the accuracy of the subsequent behavior detection result is low. SUMMARY

[0003] The present application provides at least a behavior detection method, device and storage medium.

[0004] The present application provides a behavior detection method, comprising: performing feature extraction on a to-be-detected image by using a behavior detection network to obtain target image features; performing at least one position correction on each candidate detection frame based on the target image features, and determining an updated detection frame obtained through the last position correction as a corrected detection frame of each candidate detection frame; in each position correction, determining the updated detection frame of each candidate detection frame based on current position information corresponding to each candidate detection frame and a first local image feature corresponding to the current position information in the target image features; and determining a behavior detection result of the to-be-detected image based on a second local image feature of the corrected detection frame of each candidate detection frame, the second local image feature being a feature of a corresponding region of each corrected detection frame in the target image features.

[0005] The present application provides a behavior detection device, comprising: a feature extraction module configured to perform feature extraction on a to-be-detected image by using a behavior detection network to obtain target image features; a position correction module configured to perform at least one position correction on each candidate detection frame based on the target image features, and determine an updated detection frame obtained through the last position correction as a corrected detection frame of each candidate detection frame; in each position correction, the position correction module is configured to determine the updated detection frame of each candidate detection frame based on current position information corresponding to each candidate detection frame and a first local image feature corresponding to the current position information in the target image features; and a detection module configured to determine a behavior detection result of the to-be-detected image based on a second local image feature of the corrected detection frame of each candidate detection frame, the second local image feature being a feature of a corresponding region of each corrected detection frame in the target image features.

[0006] The present application provides an electronic device comprising a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the above-mentioned behavior detection method.

[0007] The application provides a computer readable storage medium, which stores program instructions, and the program instructions are executed by a processor to realize the behavior detection method.

[0008] According to the scheme, the target image features are obtained by using the behavior detection network to perform feature extraction on the to-be-detected image, the candidate detection boxes are modified at least once by using the target image features, and then the second local image features of the modified detection boxes of the candidate detection boxes are used to determine the behavior detection result of the to-be-detected image. Compared with directly determining the second local image features corresponding to the candidate detection boxes, the second local image features obtained by the scheme are more accurate, so that the behavior detection result obtained by the scheme is more accurate.

[0009] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the application. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0011] Figure 1 is a flowchart of an embodiment of the behavior detection method of the application;

[0012] Figure 2 is a schematic diagram of an adjustment module of a cascade setting according to an embodiment of the behavior detection method of the application;

[0013] Figure 3 is a structural diagram of a behavior detection network according to an embodiment of the behavior detection method of the application;

[0014] Figure 4 is a structural diagram of an embodiment of the behavior detection device of the application;

[0015] Figure 5 is a structural diagram of an embodiment of the electronic device of the application;

[0016] Figure 6 is a structural diagram of an embodiment of the computer readable storage medium of the application. DETAILED DESCRIPTION

[0017] The scheme of the embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0018] In the following description, specific details are set forth in order to provide a thorough understanding of the application. However, persons having ordinary skill in the art will appreciate that embodiments of the application can be practiced without the specific details.

[0019] The term "and / or", used in the present document, only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in the present document generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" in the present document means two or more than two. In addition, the term "at least one" in the present document means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0020] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the behavior detection method of the present application.

[0021] As Figure 1 shown, the behavior detection method provided by the embodiments of the present disclosure can include the following steps:

[0022] Step S11: using a behavior detection network to perform feature extraction on the to-be-detected image to obtain target image features.

[0023] The to-be-detected image can be any image that needs to be detected. In some application scenarios, the to-be-detected image can be a monitoring image. In some application scenarios, the to-be-detected image can be an image obtained by a non-monitoring device. The to-be-detected image can be a two-dimensional image or a three-dimensional image. By obtaining a to-be-detected image containing a target, the behavior of the target can be analyzed to achieve the purpose of analyzing the behavior of the target. The target that needs to be analyzed can be any target, for example, the target can be a human body, an animal body, a vehicle, etc. The behavior can be any behavior that needs to be analyzed, for example, calling behavior, smoking behavior, fighting behavior, vehicle compaction line, etc. Before step S11 is executed, a step of obtaining a plurality of to-be-detected images can also be executed. The to-be-detected image can be obtained by directly photographing the target by the execution device of the behavior detection method provided by the embodiments of the present disclosure, or by photographing the target by an image acquisition device in communication connection with the execution device. The way of obtaining the sample image is not limited here.

[0024] The behavior detection network can be ResNeXt, RCNN, YOLO, SSD, etc. any model capable of feature extraction. The specific network structure of the behavior detection network is not specified here.

[0025] Step S12: based on the target image features, at least one position correction is performed on each candidate detection frame, and the updated detection frame obtained by the last position correction is determined as the corrected detection frame of each candidate detection frame.

[0026] In each position correction, the update detection frame of each candidate detection frame is determined based on the current position information corresponding to each candidate detection frame and the first local image feature corresponding to the current position information in the target image feature. The first local image feature corresponding to the current position information in the target image feature can be a region corresponding to the current position information in the target image feature determined in advance, and the features in the region are taken as the first local image feature.

[0027] For example, in the first position correction, the update detection frame of each candidate detection frame is determined based on the initial position information corresponding to each candidate detection frame and the first local image feature corresponding to the initial position information in the target image feature. In the second position correction, the position information of the update detection frame obtained in the first correction process is taken as the current position information of the candidate detection frame, and then the current position information is corrected. The same applies to other non-first position correction methods, which will not be described here.

[0028] The initial position of the target image feature before correction in the embodiments of the present disclosure can be customized by a user or obtained synchronously during the training of the behavior detection network. Alternatively, the number, size and shape of the candidate detection frame can be customized by a user or obtained synchronously during the training of the behavior detection network.

[0029] For example, the number of candidate detection frames can be set to 50, 100, 200, etc. If the number of candidate detection frames is set too high, the behavior detection process can be long and inefficient. If the number of candidate detection frames is set too low, the accuracy of behavior detection can be low. For example, the current detection algorithm needs to manually set tens of thousands of candidate detection frames in the sampling process, which results in low efficiency in setting candidate detection frames, low efficiency in sampling target image features, and low efficiency in subsequent analysis processes. The present application can achieve sparse sampling of target image features by setting a small number of candidate detection frames, thereby improving detection efficiency.

[0030] In other embodiments, the number of candidate detection frames can also be obtained by training the behavior detection network. For example, a candidate detection frame number interval can be set in advance, and the final number of candidate detection frames and the initial position information of each candidate detection frame can be determined by training the behavior detection network. By setting the number of candidate detection frames, sparse sampling of target image features can be achieved, and end-to-end sparse training can be achieved. In addition, the behavior detection network does not need to preset the position information of the candidate detection frame, and the position information can be obtained by training the behavior detection network, so that complex post-processing operations for verifying the accuracy of the classification result are not needed, and the speed and generalization of behavior detection are effectively improved.

[0031] Step S13: determining the detection result of the image to be detected based on the second local image features of the modified detection frame of each candidate detection frame.

[0032] The second local image feature is a feature of a corresponding region of the target image feature corresponding to each modified detection frame. The manner of determining the second local image feature of each modified detection frame can be to first determine the corresponding region of the target image feature corresponding to each modified detection frame, and then take the feature in the region as the second local image feature.

[0033] By detecting each second local image feature, a detection result corresponding to each second local image feature is obtained. Then, the detection result corresponding to each second local image feature can be taken as the behavior detection result of the image to be detected, or the detection results corresponding to the second local image features can be fused to obtain the behavior detection result of the image to be detected. The behavior detection result can include a result of whether a preset behavior exists in the image to be detected.

[0034] In some embodiments, the manner of obtaining the detection result corresponding to each second local image feature can also be to obtain at least one second local image feature having a correlation with the current second local image feature from the second local image features as a reference local image feature of the current second local image feature, and then obtain the detection result of the current second local image feature based on the current second local image feature and the corresponding reference local image feature. The manner of obtaining the reference local image feature of the current second local image feature can be to obtain a second local image feature having a coincidence degree, a similarity, and / or a relative distance satisfying a condition with the current second local image feature as the reference local image feature.

[0035] The above scheme uses a behavior detection network to extract features of the image to be detected to obtain the target image feature, and then modifies each candidate detection frame based on the target image feature. Then, the second local image features of the modified detection frame of each candidate detection frame are used to determine the behavior detection result of the image to be detected. Compared with directly determining the behavior detection result based on the second local image features corresponding to each candidate detection frame, the second local image features determined by the scheme are more accurate, and the determined behavior detection result is more accurate.

[0036] In some disclosed embodiments, step S12 can include the following steps: for each candidate detection frame, obtaining a target feature belonging to a target object in the first local image feature. Then, determining the correlation between the target feature and the target image feature. Then, based on the correlation, the position of the candidate detection frame is modified.

[0037] Exemplarily, the target object is a human body, the first partial image feature can include partial or all features of the human body, and can also include other background features. Features belonging to the human body are taken as target features.

[0038] Optionally, the manner of determining the correlation between the target feature and the target image feature can be: obtaining a ratio between the target feature and a reference feature. The reference feature is a feature belonging to the target object in the target image feature. Then, the ratio is taken as the correlation between the target feature and the target image feature. Specifically, the ratio between the target feature and the reference feature can be the proportion of the target feature in the reference feature, as the ratio between the target feature and the reference feature. In other words, the reference feature is taken as the foreground feature in the target image feature, and the ratio between the target feature and the reference feature is obtained as the ratio between the target feature and the foreground part in the target image feature. Exemplarily, if the target image feature includes the whole human body, and the first partial image feature only includes the head region in the human body, the ratio between the head region and the whole human body is the ratio between the target feature and the reference feature. In some other embodiments, the manner of determining the correlation between the target feature and the target image feature can also be: obtaining the ratio between the target feature and the target image feature, and taking the ratio as the correlation between the target feature and the target image feature. The specific manner of determining the correlation between the target feature and the target image feature can be determined according to user needs, which is not specified here.

[0039] In some embodiments, the behavior detection network includes a plurality of adjustment modules arranged in cascade. For example, the behavior detection network can include two or more adjustment modules arranged in cascade. The step S12 can include the following steps:

[0040] For each adjustment module, the step of determining the updated detection frame of each candidate detection frame based on the current position information of each candidate detection frame and the first partial image feature corresponding to the current position information in the target image feature is performed respectively. In the first adjustment module, the current position information of each candidate detection frame is the initial position information of each candidate detection frame. In the non-first adjustment module, the current position information of each candidate detection frame is the position information of the updated detection frame corresponding to each candidate detection frame output by the previous adjustment module. The updated detection frame of each candidate detection frame output by the last adjustment module is taken as the corrected detection frame.

[0041] In some embodiments, each adjustment module is an attention multi-head network, each attention multi-head network includes a plurality of interaction heads, each interaction head is configured to perform the step of determining the updated bounding box of each candidate bounding box based on the current position information of each candidate bounding box and the first local image feature corresponding to the current position information in the target image feature. Exemplarily, the attention multi-head network can be a spatial attention multi-head network, and can also be other types of attention multi-head networks, such as a channel attention multi-head network, a pixel attention multi-head network, etc. Optionally, the spatial attention multi-head network can be a spatial attention dynamic multi-head network.

[0042] In some disclosed embodiments, before performing the step S12, the following step can also be performed:

[0043] The current position information of each candidate bounding box is respectively extracted to obtain the current bounding box feature of each candidate bounding box. In addition, the current position information of each candidate bounding box is respectively used to determine the first local image feature corresponding to each candidate bounding box in the target image feature. On this basis, the step S12 can include the following step: for each candidate bounding box, the current bounding box feature of the candidate bounding box and the first local image feature are fused to obtain the adjusted position information of the candidate bounding box.

[0044] The position information of the candidate bounding box can be the coordinates of the four vertices of the candidate bounding box in the disclosed embodiments taking the rectangular box as an example. The current position information of each candidate bounding box is respectively extracted to obtain the current bounding box feature of each candidate bounding box, which can be a feature vector. The current position information of each candidate bounding box is respectively used to determine the first local image feature corresponding to each candidate bounding box in the target image feature, which can be to determine the region in the target image feature corresponding to the position of the candidate bounding box, and the features in the region are taken as the first local image feature corresponding to the candidate bounding box.

[0045] In which, the current bounding box feature of the candidate bounding box and the first local image feature are fused to obtain the adjusted position information of the candidate bounding box, which can be to use the spatial attention multi-head network to fuse the two to obtain the adjustment of the position of the candidate bounding box. The bounding box feature of the candidate bounding box, the position information to be adjusted of the candidate bounding box and the local image feature are fused to obtain the adjusted position information of the candidate bounding box, which can be to use the spatial attention multi-head network to fuse the three to obtain the adjustment of the position of the candidate bounding box.

[0046] For better understanding of the cascade arrangement of the adjustment modules described in the embodiments, please refer to Figure 2 , Figure 2is a schematic diagram of an adjustment module of a cascade setting shown in an embodiment of a behavior detection method. Figure 2 Three spatial attention multi-head networks with cascade settings are shown. Each spatial attention multi-head network adjusts the position information of a candidate detection box by using the detection box features obtained by feature extraction of target image features and the position information to be adjusted of the candidate detection box, to obtain adjusted position information of each candidate detection box.

[0047] In some embodiments, the behavior detection method can further include the following steps:

[0048] The behavior detection network is used to perform at least one behavior detection on each of a plurality of sample images, to obtain at least one sample detection result of each sample image. Then, based on the at least one sample detection result corresponding to each sample image, each sample image is screened to obtain a plurality of noise images. The plurality can be one or more. The sample detection result of the noise image is different from the corresponding label. Next, the behavior detection network is trained using other sample images except the plurality of noise images. In other embodiments, the behavior detection network can also be trained using noise images, thereby improving the anti-interference ability of the behavior detection network.

[0049] The above-mentioned use of the behavior detection network to perform at least one behavior detection on each of a plurality of sample images to obtain at least one sample detection result of each sample image can be that the sample image is input into the behavior detection network multiple times to obtain multiple sample detection results of the behavior detection network on the sample image.

[0050] The sample detection result includes the classification result of whether a preset behavior exists in a plurality of candidate detection boxes and the confidence corresponding to the classification result. The above-mentioned way of screening each sample image based on the at least one sample detection result corresponding to each sample image to obtain a plurality of noise images can be that for each sample detection result of each sample image, the difference between the target confidence and other confidences is determined, and the difference between the target confidence and other confidences is counted to obtain the score of the sample image with respect to the sample detection result. The target confidence is the confidence corresponding to the classification result same as the label of the sample image. Then, the scores of the sample image with respect to each sample detection result are added to obtain the final score of the sample image. In response to the final score of the sample image being less than or equal to a score threshold, the sample image is regarded as a noise image. The score of the sample image directly reflects the prediction ability of the behavior detection network on the sample image.

[0051] Optionally, the label of the sample image can specifically include labels of the target detection boxes. For example, if the label of a target detection box is a first preset value, it indicates that the preset behavior exists in the target detection box, and if the label of the target detection box is a second preset value, it indicates that the preset behavior does not exist in the target detection box. Wherein, for each sample image, the labels of the target detection boxes corresponding to the sample image can be the same or different. For example, there are multiple targets in the sample image to be detected for behavior, and the behaviors of each target are different, which can result in different labels of the target detection boxes. When calculating the score of the sample image, the scores corresponding to the behavior detection results of different targets can be calculated respectively. Then, the scores of different targets are added, or averaged, or the maximum value is taken, or any other statistical method to obtain the score of the sample image.

[0052] For example, in the sample image a, there is one target, which has the preset behavior, and 5 candidate detection boxes are preset. Therefore, there are 5 classification results, and only one of them has the same label as the sample image, and the other four have opposite labels. Therefore, the confidence of the classification result with the same label as the sample image is subtracted from the other confidence, and the difference is calculated to obtain the score of the sample image. In some embodiments, the maximum difference can be taken as the score of the sample image, and in other embodiments, the average difference, the median, the mode, or any other statistical value can be taken as the score of the sample image. In the present embodiment, the maximum difference is taken as the score of the sample image.

[0053] In the above example, if the behavior detection network performs 3 behavior detections on the sample image a, and obtains 3 sample detection results, the score of the sample image a with respect to each sample detection result is determined based on the above method, and the final score of the sample image is obtained by adding the 3 scores. The score threshold can be determined according to specific requirements. Optionally, multiple score thresholds can be set, and the sample images can be classified into easy samples, difficult example samples, and noise samples according to the score thresholds. For example, if the score is large, it indicates that the prediction ability for the sample image is strong, and the sample image is determined as an easy sample. If the score is small, it indicates that the prediction ability for the sample image is weak, and the sample image is determined as a noise sample. If the score is close to the middle value, it indicates that the prediction ability of the behavior detection network for the sample image is weak, and the sample image is determined as a difficult example sample.

[0054] In some embodiments, the difficult example samples and the noise samples can be used to train the behavior detection network to improve the accuracy and generalization of the behavior detection network. In some embodiments, the sample images with scores less than or equal to the score threshold can be deleted, and only the sample images with scores greater than the score threshold are retained for subsequent training of the behavior detection network. In this way, the interference samples are removed, and the robustness of the model can be improved.

[0055] In some embodiments, the initial position information of each candidate detection frame before position correction is obtained by training the behavior detection network. The sample detection result includes a classification result of whether a preset behavior exists in a plurality of candidate detection frames, a confidence corresponding to the classification result, and a position of each candidate detection frame in the corresponding sample image. The behavior detection method can further include the following steps: determining a symmetric cross-entropy loss by using the classification result corresponding to each sample image, and determining a regression loss by using the adjusted position of the candidate detection frame. Then, based on the symmetric cross-entropy loss and the regression loss, the parameters in the behavior detection network are adjusted, and the position information of the candidate detection frame is adjusted. The adjusted position of the candidate detection frame can be the position output by the last level adjustment module. In the first training process of the behavior detection network, the position information of the candidate detection frame used is preset position information, and the position information of the candidate detection frame obtained after training is updated to be the initial position information of each candidate detection frame before position correction.

[0056] The symmetric cross-entropy loss can be obtained by referring to formulas (1) to (3):

[0057]

[0058]

[0059] l sce =l rce +l ce (3);

[0060] wherein l sce represents the symmetric cross-entropy loss, l rce represents a first loss, l ce represents a second loss, the first loss and the second loss are added to obtain the cross-entropy loss, K represents a classification result, k = 1 represents that the classification result is a classification result of existing a preset behavior. x represents an identifier of a sample image, p represents a confidence of each candidate detection of the sample image output by the behavior detection network, and q represents a label.

[0061] In some embodiments, the candidate frame position in the sample detection result output by the behavior detection network is matched with a target detection frame to obtain a matching group, and a regression loss between the candidate detection frame and the target detection frame in the matching group is determined. The target detection frame is a pre-set label.

[0062] For better understanding of the behavior detection network provided in the present embodiment, please refer to Figure 3 , Figure 3 is a structural schematic diagram of a behavior detection network according to an embodiment of the behavior detection method of the present application. As shown in Figure 3As shown, the behavior detection network includes a backbone feature extraction network for performing the above step S11 to output target image features. The initial position information of the candidate detection frame is extracted through a fully connected layer to extract frame features. In addition, the corresponding local image features are determined by combining the initial position information of the candidate detection frame with the region feature aggregation method (for example, RoI Align). The local image features are pooled to a uniform size through RoI Align. Then, the detection frame features interact with the corresponding first local image features through a spatial attention mechanism to output the position information of the adjusted candidate detection frame. In addition, the first local image features and the candidate detection frame features are interacted through the use of the spatial attention mechanism, which can enhance the foreground features. The position information of the candidate detection frame output by the last spatial attention multi-head network is used to obtain the corresponding second local image features, and the obtained second local image features are input into a 2-layer fully connected layer to obtain the corresponding behavior detection result, including the class, confidence and position of the candidate detection frame.

[0063] The above scheme uses the behavior detection network to extract features from the image to be detected to obtain target image features, and then uses the target image features to correct each candidate detection frame at least once, and then uses the second local image features of the corrected detection frame of each candidate detection frame to determine the behavior detection result of the image to be detected. Compared with directly determining the behavior detection result based on the second local image features corresponding to each candidate detection frame, the second local image features obtained by the above scheme are more accurate, so that the behavior detection result obtained by the above scheme is more accurate.

[0064] The execution subject of the behavior detection method can be a behavior detection device, for example, the behavior detection device can be a terminal device or a server or other processing device, wherein the terminal device can be a monitoring device in a security system, a network video recorder, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the behavior detection method can be realized by a processor calling computer readable instructions stored in a memory.

[0065] Please refer to Figure 4 , Figure 4is a structural schematic diagram of an embodiment of a behavior detection apparatus. The behavior detection apparatus 30 includes a feature extraction module 31, a position correction module 32, and a detection module 33. The feature extraction module 31 is configured to perform feature extraction on a to-be-detected image by using a behavior detection network to obtain target image features. The position correction module 32 is configured to perform at least one position correction on each candidate detection box based on the target image features, and determine an updated detection box obtained through the last position correction as a corrected detection box of each candidate detection box. In each position correction, an updated detection box of each candidate detection box is determined based on current position information corresponding to each candidate detection box and a first local image feature corresponding to the current position information in the target image features. The detection module 33 is configured to determine a behavior detection result of the to-be-detected image based on a second local image feature of the corrected detection box of each candidate detection box, the second local image feature being a feature of a region corresponding to each corrected detection box in the target image features.

[0066] The above scheme, after performing feature extraction on a to-be-detected image by using a behavior detection network to obtain target image features, performing at least one correction on each candidate detection box by using the target image features, and then determining a behavior detection result of the to-be-detected image based on a second local image feature of a corrected detection box of each candidate detection box, the second local image feature is more accurate than a second local image feature corresponding to each candidate detection box, so that the determined behavior detection result is more accurate.

[0067] The functions of each module can be referred to the embodiments of the behavior detection method, which will not be repeated here.

[0068] Please refer to Figure 5 , Figure 5 is a structural schematic diagram of an embodiment of an electronic device. The electronic device 40 includes a memory 41 and a processor 42, and the processor 42 is configured to execute program instructions stored in the memory 41 to implement the steps in any of the above behavior detection method embodiments. In a specific implementation scenario, the electronic device 40 can include but is not limited to a monitoring device, a microcomputer, a server, and in addition, the electronic device 40 can also include a notebook computer, a tablet computer, and other mobile devices, which are not limited here.

[0069] Specifically, the processor 42 is configured to control itself and the memory 41 to implement the steps in any of the above-described behavior detection method embodiments. The processor 42 can also be referred to as a CPU (Central Processing Unit). The processor 42 can be an integrated circuit chip having a processing capability of signals. The processor 42 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like. In addition, the processor 42 can be implemented by an integrated circuit chip together.

[0070] The above scheme, by utilizing the behavior detection network to perform feature extraction on the to-be-detected image, obtaining the target image feature, then performing at least one correction on each candidate detection box by using the target image feature, and then determining the behavior detection result of the to-be-detected image by using the second local image feature of the corrected detection box of each candidate detection box, compared with directly based on the second local image feature corresponding to each candidate detection box, the second local image feature determined by the scheme is more accurate, so that the behavior detection result determined is more accurate.

[0071] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of an embodiment of the computer readable storage medium of the present application. The computer readable storage medium 50 stores program instructions 51 capable of being executed by a processor, and the program instructions 51 are used to implement the steps in any of the above-described behavior detection method embodiments.

[0072] The above scheme, by utilizing the behavior detection network to perform feature extraction on the to-be-detected image, obtaining the target image feature, then performing at least one correction on each candidate detection box by using the target image feature, and then determining the behavior detection result of the to-be-detected image by using the second local image feature of the corrected detection box of each candidate detection box, compared with directly based on the second local image feature corresponding to each candidate detection box, the second local image feature determined by the scheme is more accurate, so that the behavior detection result determined is more accurate.

[0073] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be described here.

[0074] The above description of the various embodiments tends to emphasize differences between the various embodiments, and the same or similar elements can be referred to each other, and will not be described herein for the sake of brevity.

[0075] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the above-described device implementation is only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a unit or component can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed elements can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0076] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or say the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method of behavior detection, the method comprising: The method comprises: feature extraction is performed on the to-be-detected image by using a behavior detection network to obtain target image features; a plurality of position corrections are performed on each candidate detection box based on the target image features, and an updated detection box obtained through the last position correction is determined as a corrected detection box of each candidate detection box; the behavior detection network comprises a plurality of adjustment modules arranged in cascade, each adjustment module determines an updated detection box of each candidate detection box based on current position information of each candidate detection box and a first local image feature corresponding to the current position information in the target image features; wherein the current position information of each candidate detection box in the first adjustment module is initial position information of each candidate detection box, the current position information of each candidate detection box in a non-first adjustment module is position information of an updated detection box corresponding to each candidate detection box output by an upper adjustment module, and the updated detection box of each candidate detection box output by the last adjustment module is taken as the corrected detection box of each candidate detection box; a behavior detection result of the to-be-detected image is determined based on a second local image feature of the corrected detection box of each candidate detection box, the second local image feature being a feature of a corresponding region of each corrected detection box in the target image features.

2. The method of claim 1, wherein, The method comprises: for each candidate detection box, target features belonging to a target object in the first local image feature are obtained; correlation between the target features and the target image features is determined; the position of the candidate detection box is corrected based on the correlation.

3. The method of claim 2, wherein, The method comprises: a ratio between the target features and reference features is obtained, the reference features being features belonging to the target object in the target image features; the ratio is taken as the correlation between the target features and the target image features.

4. The method according to any one of claims 1 to 3, characterized in that, Before the updated detection box of each candidate detection box is determined based on the current position information of each candidate detection box and the first local image feature corresponding to the current position information in the target image features, the method comprises: feature extraction is performed on the current position information of each candidate detection box to obtain current detection box features of each candidate detection box; and the first local image feature corresponding to the current position information of each candidate detection box in the target image features is determined by using the current position information of each candidate detection box. The method comprises: for each candidate detection box, the current detection box features of the candidate detection box and the first local image feature are fused to obtain adjusted position information of the candidate detection box.

5. The method of claim 4, wherein, Each of the adjustment modules comprises an attention multi-head network, and each of the attention multi-head networks comprises a plurality of interaction heads, and each of the interaction heads is configured to perform, for one of the candidate bounding boxes, the step of determining the updated bounding box of each of the candidate bounding boxes based on the current position information corresponding to the candidate bounding box and the first local image feature corresponding to the current position information in the target image feature.

6. The method of claim 1, wherein, The method further comprises a training step of the behavior detection network, and the training step comprises: performing, by using the behavior detection network, behavior detection on a plurality of sample images at least once to obtain at least one sample detection result of each of the sample images; performing screening on each of the sample images based on at least one sample detection result corresponding to the sample image to obtain a plurality of noise images, wherein the sample detection result of the noise image is different from a corresponding label; training the behavior detection network by using sample images other than the plurality of noise images.

7. The method of claim 6, wherein, The sample detection result comprises a classification result of whether a preset behavior exists in a plurality of candidate bounding boxes and a confidence corresponding to the classification result, and the screening of each of the sample images based on at least one sample detection result corresponding to the sample image to obtain a plurality of noise images comprises: for each of the sample detection results of each of the sample images, determining a difference between a target confidence and other confidences, and counting the difference between the target confidence and the other confidences to obtain a score of the sample image with respect to the sample detection result, wherein the target confidence is a confidence corresponding to a classification result same as a label of the sample image; adding the scores corresponding to each of the sample detection results of the sample image to obtain a final score of the sample image; in response to the final score of the sample image being less than or equal to a score threshold, regarding the sample image as the noise image.

8. The method of claim 6, wherein, The initial position information of each of the candidate bounding boxes before position correction is obtained by training the behavior detection network, the sample detection result comprises a classification result of whether a preset behavior exists in a plurality of candidate bounding boxes, a confidence corresponding to the classification result, and a position of each of the candidate bounding boxes in a corresponding sample image, and the method further comprises: determining a symmetric cross-entropy loss by using the classification result corresponding to each of the sample images, and determining a regression loss by using the adjusted position of the candidate bounding box; adjusting parameters in the behavior detection network and adjusting the position information of the candidate bounding box based on the symmetric cross-entropy loss and the regression loss; wherein the position information of the candidate bounding box used in the first training process of the behavior detection network is preset position information, and the position information of the candidate bounding box updated after the last training of the behavior detection network is the initial position information.

9. An electronic device, comprising: The device comprises a memory and a processor, and the processor is configured to execute program instructions stored in the memory to implement the method in any one of claims 1 to 8.

10. A computer-readable storage medium having stored thereon program instructions, wherein, The program instructions are executed by the processor to implement the method in any one of claims 1 to 8. The program instructions are executed by the processor to implement the method in any one of claims 1 to 8.

Images