Object detection method and apparatus, electronic device, and storage medium

Abstract

The application provides an object detection method and device, electronic equipment and storage medium, and relates to the field of biological identification. The method comprises the following steps: acquiring a first video collected by a camera in a preset area; determining multiple face images and first behavior information corresponding to each face image in the first video; acquiring M second behavior information corresponding to each face image in a preset database, the preset database is used for storing behavior information corresponding to each face image appearing in the preset area in a historical period, and M is an integer; determining a target face image in the multiple face images according to the first behavior information and the M second behavior information corresponding to each face image, and a probability that an object corresponding to the target face image performs a preset behavior is greater than or equal to a preset probability. According to the above scheme, the behavior information is introduced as a detection basis, which is a supplement to face image recognition, object detection can be performed from multiple dimensions, and the accuracy of object detection is improved.

Description

Technical Field

[0001] This application relates to the field of biometrics, and more particularly to an object detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] In bank branches or other locations with high security requirements, adequate security measures must be in place to address potential risks and ensure personal and property safety. Simultaneously, establishing sufficient early warning systems allows for proactive prevention of suspicious individuals before risks materialize, effectively maintaining order and security while enhancing the quality of banking services.

[0003] In related technologies, historical risk objects can be identified. These are objects with a history of risk. By comparing the facial images of each object in the venue with the facial images of objects with a history of risk, the presence of suspicious objects in the venue can be detected.

[0004] However, the above methods can only detect objects with risk records, and cannot accurately detect objects without risk records. Summary of the Invention

[0005] This application provides an object detection method, apparatus, electronic device, and storage medium to improve the accuracy of object detection.

[0006] In a first aspect, this application provides an object detection method, comprising: acquiring a first video captured by a camera device within a preset area; determining multiple face images and first behavioral information corresponding to each face image in the first video, the first behavioral information including a movement trajectory and a movement duration; acquiring M second behavioral information corresponding to each face image in a preset database, the preset database being used to store behavioral information corresponding to each face image appearing in the preset area within a historical time period, the M being an integer, the second behavioral information including a movement trajectory and a movement duration; determining a target face image among the multiple face images based on the first behavioral information and the M second behavioral information corresponding to each face image, wherein the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to a preset probability.

[0007] In one possible implementation, determining a target face image from the plurality of face images based on first behavioral information and M second behavioral information corresponding to each face image includes: determining at least one candidate face image from the plurality of face images based on the first behavioral information corresponding to each face image, wherein the action duration corresponding to the candidate face image is greater than or equal to a preset duration; and determining the target face image from the at least one candidate face image based on the M second behavioral information corresponding to each candidate face image.

[0008] In one possible implementation, determining the target face image from the at least one candidate face image based on M second behavioral information corresponding to each candidate face image includes: determining at least one selected face image from the plurality of candidate face images based on the number M of second behavioral information corresponding to each candidate face image and the action duration in each behavioral information, wherein the number M of second behavioral information corresponding to the selected face image is greater than or equal to a preset number, and the action duration in each behavioral information corresponding to the selected face image is greater than or equal to a preset duration; and processing each selected face image, as well as the first behavioral information and M second behavioral information corresponding to each selected face image, through a target model to determine the target face image from the at least one candidate face image.

[0009] In one possible implementation, a target model is used to process each selected face image, as well as the first behavioral information and M second behavioral information corresponding to each selected face image, to determine the target face image among the at least one candidate face image. This includes: processing each selected face image, as well as the first behavioral information and M second behavioral information corresponding to each selected face image, using the target model to obtain a predicted probability corresponding to each selected face image, wherein the predicted probability is the probability that the object corresponding to the selected face image will perform the preset behavior; and determining the selected face images whose predicted probability is greater than or equal to the preset probability as the target face images.

[0010] In one possible implementation, the target model includes an image feature extraction layer, a behavior feature extraction layer, a fusion layer, and a classification layer. For any selected face image, the target model processes the selected face image, along with first behavior information and M second behavior information corresponding to the selected face image, to obtain a predicted probability corresponding to the selected face image. This process includes: processing the selected face image through the image feature extraction layer to obtain face features; processing the first behavior information and M second behavior information corresponding to the selected face image through the behavior feature extraction layer to obtain behavior features; fusing the face features and behavior features through the fusion layer to obtain fused features; and processing the fused features through the classification layer to obtain the predicted probability.

[0011] In one possible implementation, multiple face images and first behavioral information corresponding to each face image are determined in the first video. The first behavioral information includes movement trajectory and movement duration. This includes: performing face recognition processing on each image frame in the first video to obtain the multiple face images; for any face image, determining the position of the face image at multiple times based on the position of the face image in each image frame in the first video and the installation position of the camera device, and determining the first behavioral information corresponding to the face image based on the position of the face image at the multiple times, wherein the multiple times correspond to the shooting time of each image frame in the first video.

[0012] In one possible implementation, determining the first behavioral information corresponding to the face image based on the position of the face image at the plurality of times includes: determining the action trajectory corresponding to the face image based on the position of the face image at the plurality of times; determining the start time and end time of the face image in the first video; and determining the difference between the end time and the start time as the action duration corresponding to the face image.

[0013] In one possible implementation, the method further includes: storing the plurality of face images, as well as the first behavioral information and M second behavioral information corresponding to each face image, into the preset database.

[0014] Secondly, this application provides an object detection device, comprising: an acquisition module for acquiring a first video captured by a camera device within a preset area; a determination module for determining multiple face images and first behavioral information corresponding to each face image in the first video, the first behavioral information including a movement trajectory and a movement duration; a query module for acquiring M second behavioral information corresponding to each face image from a preset database, the preset database being used to store behavioral information corresponding to each face image appearing in the preset area within a historical time period, M being an integer, the second behavioral information including a movement trajectory and a movement duration; and a detection module for determining a target face image among the multiple face images based on the first behavioral information and the M second behavioral information corresponding to each face image, wherein the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to a preset probability.

[0015] In one possible implementation, the detection module is specifically configured to determine at least one candidate face image among the plurality of face images based on first behavioral information corresponding to each face image, wherein the action duration corresponding to the candidate face image is greater than or equal to a preset duration; the detection module is further configured to determine the target face image among the at least one candidate face image based on M second behavioral information corresponding to each candidate face image.

[0016] In one possible implementation, the detection module is specifically configured to determine at least one selected face image from the plurality of candidate face images based on the number M of second behavioral information corresponding to each candidate face image and the action duration in each behavioral information, wherein the number M of second behavioral information corresponding to the selected face image is greater than or equal to a preset number, and the action duration in each behavioral information corresponding to the selected face image is greater than or equal to a preset duration; the detection module is further configured to process each selected face image, and the first behavioral information and M second behavioral information corresponding to each selected face image through a target model, so as to determine the target face image from the at least one candidate face image.

[0017] In one possible implementation, the detection module is specifically used to process each selected face image, as well as the first behavioral information and M second behavioral information corresponding to each selected face image, through a target model to obtain a predicted probability corresponding to each selected face image, wherein the predicted probability is the probability that the object corresponding to the selected face image will perform the preset behavior; the detection module is further specifically used to determine the selected face images whose predicted probability is greater than or equal to the preset probability as the target face images.

[0018] In one possible implementation, the target model includes an image feature extraction layer, a behavior feature extraction layer, a fusion layer, and a classification layer. For any selected face image, the detection module is specifically configured to process the selected face image through the image feature extraction layer to obtain face features; the detection module is further configured to process the first behavior information and M second behavior information corresponding to the selected face image through the behavior feature extraction layer to obtain behavior features; the detection module is further configured to fuse the face features and the behavior features through the fusion layer to obtain fused features; and the detection module is further configured to process the fused features through the classification layer to obtain the predicted probability.

[0019] In one possible implementation, the device further includes: a recognition module, configured to perform face recognition processing on each image frame in the first video to obtain the plurality of face images; the recognition module is further configured to, for any face image, determine the position of the face image at multiple times based on the position of the face image in each image frame in the first video and the installation position of the camera device, and determine the first behavioral information corresponding to the face image based on the position of the face image at the multiple times, wherein the multiple times correspond to the shooting time of each image frame in the first video.

[0020] In one possible implementation, the recognition module is specifically configured to determine the movement trajectory corresponding to the face image based on the position of the face image at the multiple times; the recognition module is further configured to determine the start time and end time of the face image in the first video; the recognition module is further configured to determine the difference between the end time and the start time as the action duration corresponding to the face image.

[0021] In one possible implementation, the device further includes a storage module for storing the plurality of face images, as well as first behavioral information and M second behavioral information corresponding to each face image, into the preset database.

[0022] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described in any one of the first aspects.

[0023] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which are executed by a processor as described in any one of the first aspects.

[0024] The object detection method, apparatus, electronic device, and storage medium provided in this application include: acquiring a first video captured by a camera device within a preset area; determining multiple face images and first behavioral information corresponding to each face image in the first video, the first behavioral information including movement trajectory and action duration; acquiring M second behavioral information corresponding to each face image in a preset database, the preset database being used to store behavioral information corresponding to each face image appearing in the preset area within a historical time period, where M is an integer, and the second behavioral information including movement trajectory and action duration; determining a target face image among the multiple face images based on the first behavioral information and the M second behavioral information corresponding to each face image, wherein the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to a preset probability. This scheme introduces behavioral information as a detection basis, supplementing face image recognition, and can perform object detection from multiple dimensions, thereby improving the accuracy of object detection. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0026] Figure 1 This is a schematic diagram illustrating an application scenario of an object detection method provided in an embodiment of this application;

[0027] Figure 2 A flowchart illustrating an object detection method provided in an embodiment of this application;

[0028] Figure 3 A flowchart illustrating an object detection method provided in an embodiment of this application;

[0029] Figure 4 This is a schematic diagram illustrating the determination of the first line of information provided in an embodiment of this application.

[0030] Figure 5 This is a schematic diagram illustrating the calculation and prediction probability provided in an embodiment of this application.

[0031] Figure 6 This is a schematic diagram of model training provided in an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of the structure of an object detection device provided in an embodiment of this application;

[0033] Figure 8 This is a schematic diagram of the structure of an object detection device provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0035] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0038] It should be noted that the object detection method and apparatus of this application can be used in the field of biometrics, or in any field other than biometrics. The application field of the object detection method and apparatus of this application is not limited.

[0039] Figure 1 This is a schematic diagram illustrating an application scenario of an object detection method provided in an embodiment of this application. An example is given based on the illustrated scenario: a preset area includes multiple objects; a camera device captures video within the preset area, the captured video including each object; the captured video is sent to a backend system; and the backend system determines the target object from the captured video.

[0040] The data transmission method provided in this application is intended to solve the above-mentioned technical problems of the prior art.

[0041] The technical solutions of this application will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In the description of this application, unless otherwise expressly specified and limited, the terms should be broadly understood within the art. The embodiments of this application will now be described with reference to the accompanying drawings.

[0042] Figure 2This is a flowchart illustrating an object detection method provided in an embodiment of this application. The method includes the following steps:

[0043] S201. Acquire the first video captured by the camera device within the preset area.

[0044] As an example, the execution subject of this embodiment can be an object detection device, which can be implemented in various ways. For example, it can be program software, or a medium storing relevant computer programs, such as a USB flash drive; or, the device can also be a physical device that integrates or installs relevant computer programs, such as a chip, a smart terminal, a computer, a server, etc.

[0045] The first video includes timestamps and timeline information.

[0046] For example, the timestamp is the moment when video capture begins, and the timeline information includes the time point corresponding to each frame of the video.

[0047] Optionally, the capture time of each frame in the video can be determined using timestamps and timeline information.

[0048] S202. In the first video, determine multiple face images and first behavioral information corresponding to each face image, the first behavioral information including movement trajectory and movement duration.

[0049] Each facial image corresponds to an object to be filtered.

[0050] Optionally, the movement trajectory can be generated into a path map, which includes multiple locations traversed by the object to be filtered, as well as the time of passing through each location.

[0051] S203. Obtain M second behavioral information corresponding to each face image from a preset database. The preset database is used to store behavioral information corresponding to each face image that appears in the preset area within a historical time period. M is an integer. The second behavioral information includes the movement trajectory and the movement duration.

[0052] For example, behavioral information corresponding to all facial images appearing in the preset area during a historical period is stored in a preset database.

[0053] It is understandable that by using a pre-set database, the frequency of each object appearing in a pre-set area can be determined.

[0054] Optionally, behavioral information for a preset time period is stored in the preset database, and the behavioral information is deleted from the oldest to the newest after the preset time period has expired.

[0055] Alternatively, a maximum number of behavioral information entries can be saved in the preset database. If the number of saved behavioral information entries exceeds the maximum, the entries will be deleted from the furthest to the nearest, or the entries will be manually deleted by staff.

[0056] Alternatively, the plurality of face images, as well as the first line information and M second line information corresponding to each face image, are stored in the preset database.

[0057] In terms of distance, the default database is updated in real time.

[0058] S204. Based on the first behavior information and M second behavior information corresponding to each face image, determine the target face image among the plurality of face images, wherein the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to the preset probability.

[0059] Among them, the preset behavior can be a behavior that poses a risk to the preset area.

[0060] Optionally, objects whose second behavior occurs frequently and whose first behavior is abnormal are identified as target objects, and their facial images are identified as target facial images.

[0061] For example, if an object frequently performs risky behaviors in a preset area, then that object is identified as the target object.

[0062] The object detection method provided in this application involves acquiring a first video captured by a camera device within a preset area; identifying multiple face images and first behavioral information corresponding to each face image in the first video, the first behavioral information including movement trajectory and action duration; acquiring M second behavioral information corresponding to each face image from a preset database, the preset database being used to store behavioral information corresponding to each face image appearing in the preset area within a historical time period, where M is an integer, and the second behavioral information including movement trajectory and action duration; and determining a target face image from the multiple face images based on the first behavioral information and the M second behavioral information, wherein the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to a preset probability. This scheme introduces behavioral information as a detection basis, supplementing face image recognition, and can perform object detection from multiple dimensions, thereby improving the accuracy of object detection.

[0063] Based on any of the above embodiments, the following, in conjunction with Figure 3 The detailed process of object detection is explained.

[0064] Figure 3 This is a flowchart illustrating an object detection method provided in an embodiment of this application. Figure 3 As shown, the method includes:

[0065] S301. Acquire the first video captured by the camera device within the preset area.

[0066] It should be noted that the execution process of S301 is the same as that of S201, and will not be repeated here.

[0067] S302. Perform face recognition processing on each image frame in the first video to obtain the plurality of face images.

[0068] Optionally, a face image can be identified from each image frame using a face recognition algorithm.

[0069] Alternatively, the face image may be preprocessed, including but not limited to: image noise reduction, image enhancement, image sharpening, and exposure adjustment.

[0070] S303. For any face image, determine the position of the face image at multiple times based on the position of the face image in each image frame of the first video and the installation position of the camera device, and determine the first behavioral information corresponding to the face image based on the position of the face image at the multiple times, wherein the multiple times correspond to the shooting time of each image frame in the first video.

[0071] To illustrate with a scenario example, a schematic diagram of a preset area and the installation location of the camera device are predetermined. The positions of the face image in each image frame of the first video are compared with the schematic diagram of the preset area to determine the positions of the face image in the schematic diagram of the preset area at multiple times.

[0072] One feasible implementation involves determining the first behavioral information corresponding to the face image based on its position at multiple times, including: determining the action trajectory corresponding to the face image based on its position at multiple times; determining the start time and end time of the face image in the first video; and determining the difference between the end time and the start time as the action duration corresponding to the face image.

[0073] Below, in conjunction with Figure 4 Explain the information used to determine the first line.

[0074] Figure 4 This is a schematic diagram illustrating the determination of the first line of information provided in an embodiment of this application. For example... Figure 4 As shown, a motion trajectory is generated based on the position of the face image at each time step. The motion trajectory includes the sequence of the face image's movements. The starting time is time 1, and the ending time is time 8. The difference between time 8 and time 1 is calculated to obtain the motion duration corresponding to the face image.

[0075] In this feasible implementation, the movement trajectory and duration are determined by the position of the face image at the multiple times, which can accurately reflect the object's movement information, thereby improving the accuracy of object detection.

[0076] S304. Obtain M second behavioral information corresponding to each face image from a preset database. The preset database is used to store behavioral information corresponding to each face image that appears in the preset area within a historical time period. M is an integer, and the second behavioral information includes the movement trajectory and the movement duration.

[0077] It should be noted that the execution process of S304 is the same as that of S203 and will not be repeated here.

[0078] S305. Based on the first behavior information corresponding to each face image, determine at least one candidate face image among the plurality of face images, wherein the action duration corresponding to the candidate face image is greater than or equal to a preset duration.

[0079] In a scenario example, assuming the preset duration is 10 minutes, if the action duration is greater than or equal to 10 minutes, the corresponding face image will be identified as a candidate face image. The subsequent face images need to be processed through further segments to determine the target face image.

[0080] S306. Based on the number M of second behavioral information corresponding to each candidate face image and the action duration in each behavioral information, at least one selected face image is determined among the plurality of candidate face images. The number M of second behavioral information corresponding to the selected face image is greater than or equal to a preset number, and the action duration in each behavioral information corresponding to the selected face image is greater than or equal to a preset duration.

[0081] In a scenario example, assuming the preset quantity is 10, if the quantity M of the second action information is greater than or equal to 10, and the action duration of each action information is greater than or equal to the preset duration, it means that the face image has frequently appeared in the preset area recently, and each time it appears in the preset area, it lasts for a period of time. Then, it is determined as the selected face image.

[0082] Optionally, a filtering period can be set, and the selected face image can be determined by the second row of information within the filtering period.

[0083] For example, if the filtering period is one week and the number of second-behavior information within the past week is N, if N is greater than or equal to the preset number, and the action duration of each of the N behavior information is greater than or equal to the preset duration, then the corresponding candidate face image is determined as the selected face image.

[0084] S307. Process each selected face image, as well as the first action information and M second action information corresponding to each selected face image, through the target model to obtain the predicted probability corresponding to each selected face image. The predicted probability is the probability that the object corresponding to the selected face image will perform the preset action.

[0085] The target model includes an image feature extraction layer, a behavior feature extraction layer, a fusion layer, and a classification layer.

[0086] Optionally, the image feature extraction layer uses a convolutional neural network, the behavior feature extraction layer uses a graph convolutional neural network, and the classification layer uses a support vector machine algorithm.

[0087] A feasible implementation, for any selected face image, involves obtaining the predicted probability corresponding to the selected face image, including: processing the selected face image through the image feature extraction layer to obtain face features; processing the first behavioral information and M second behavioral information corresponding to the selected face image through the behavioral feature extraction layer to obtain behavioral features; fusing the face features and the behavioral features through the fusion layer to obtain fused features; and processing the fused features through the classification layer to obtain the predicted probability.

[0088] Below, in conjunction with Figure 5 The calculation of the predicted probability is explained.

[0089] Figure 5 This is a schematic diagram illustrating the calculation and prediction probability provided in an embodiment of this application. For example... Figure 5 As shown, the selected face image is input into the image feature extraction layer to obtain face features. The behavioral information corresponding to the selected face image is input into the behavioral feature extraction layer to obtain behavioral features. The face features and behavioral features are input into the fusion layer to obtain fused features. The fused features are input into the classification layer to obtain the predicted probability. The higher the preset probability, the higher the probability that the object corresponding to the face image will perform the preset behavior.

[0090] Optionally, the target model is trained using historical face images and the corresponding behavioral information.

[0091] Below, in conjunction with Figure 6 The model training process will be explained.

[0092] Figure 6 This is a schematic diagram of model training provided for an embodiment of this application. For example... Figure 6As shown, historical face images and their corresponding historical behavior information are obtained from a pre-set database. Behavior annotations for the historical face images are determined manually, including whether the historical face image performed a preset behavior or not. Using the historical face images and historical behavior information as input, and the behavior annotations as labels, the model is trained to obtain the target model.

[0093] In this feasible implementation, prediction is made by combining two dimensions, facial features and behavioral features, through the target model. Compared with prediction based on a single dimension, the prediction accuracy is higher.

[0094] S308. Selected face images with a predicted probability greater than or equal to the preset probability are determined as the target face images.

[0095] Optionally, the preset probability can be modified according to the business scenario.

[0096] In a scenario example, if the predicted probability is greater than or equal to the preset probability, it means that the behavioral information of the selected face image matches the characteristics of the preset behavior, and the selected face image is determined as the target face image.

[0097] Figure 7 This is a schematic diagram of the structure of an object detection device provided in an embodiment of this application. Figure 7 As shown, the object detection device 70 may include: an acquisition module 71, a determination module 72, a query module 73, and a detection module 74, wherein,

[0098] The acquisition module 71 is used to acquire the first video captured by the camera device within a preset area.

[0099] The determining module 72 is used to determine multiple face images in the first video and first behavioral information corresponding to each face image, the first behavioral information including movement trajectory and movement duration.

[0100] The query module 73 is used to obtain M second behavioral information corresponding to each face image from a preset database. The preset database is used to store behavioral information corresponding to each face image that appears in the preset area within a historical time period. M is an integer, and the second behavioral information includes the movement trajectory and the movement duration.

[0101] The detection module 74 is used to determine a target face image among the plurality of face images based on the first behavior information and M second behavior information corresponding to each face image, wherein the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to a preset probability.

[0102] Optionally, module 71 can be executed. Figure 2S201 in the embodiment.

[0103] Optionally, module 72 can be executed. Figure 2 S202 in the embodiment.

[0104] Optionally, query module 73 can execute... Figure 2 S203 in the embodiment.

[0105] Optionally, the detection module 74 can perform... Figure 2 S204 in the embodiment.

[0106] It should be noted that the object detection device shown in the embodiments of this application can execute the technical solution shown in the above method embodiments, and its implementation principle and beneficial effects are similar, so they will not be described again here.

[0107] In one possible implementation, the detection module 74 is specifically used for:

[0108] Based on the first behavior information corresponding to each face image, at least one candidate face image is determined from the plurality of face images, and the action duration corresponding to the candidate face image is greater than or equal to a preset duration.

[0109] The target face image is determined from the at least one candidate face image based on M second-line information corresponding to each candidate face image.

[0110] In one possible implementation, the detection module 74 is specifically used for:

[0111] Based on the number M of second behavioral information corresponding to each candidate face image and the action duration in each behavioral information, at least one selected face image is determined among the plurality of candidate face images. The number M of second behavioral information corresponding to the selected face image is greater than or equal to a preset number, and the action duration in each behavioral information corresponding to the selected face image is greater than or equal to a preset duration.

[0112] The target face image is determined by processing each selected face image, as well as the first line information and M second line information corresponding to each selected face image, through the target model.

[0113] In one possible implementation, the detection module 74 is specifically used for:

[0114] The target model processes each selected face image, as well as the first action information and M second action information corresponding to each selected face image, to obtain the predicted probability corresponding to each selected face image. The predicted probability is the probability that the object corresponding to the selected face image will perform the preset action.

[0115] Selected face images whose predicted probability is greater than or equal to the preset probability are identified as the target face images.

[0116] In one possible implementation, the target model includes an image feature extraction layer, a behavior feature extraction layer, a fusion layer, and a classification layer; for any selected face image; the detection module 74 is specifically used for:

[0117] The selected face image is processed by the image feature extraction layer to obtain face features;

[0118] The behavior feature extraction layer processes the first behavior information and M second behavior information corresponding to the selected face image to obtain behavior features;

[0119] The facial features and behavioral features are fused together using the fusion layer to obtain fused features;

[0120] The fused features are processed by the classification layer to obtain the predicted probability.

[0121] Figure 8 This is a schematic diagram of the structure of an object detection device provided in an embodiment of this application. Figure 7 Based on the illustrated embodiments, as Figure 8 As shown, the object detection device 80 further includes: an identification module 75 and a storage module 76, wherein:

[0122] The identification module 75 is used for:

[0123] Face recognition processing is performed on each image frame in the first video to obtain the plurality of face images;

[0124] For any given face image, the position of the face image at multiple times is determined based on the position of the face image in each image frame of the first video and the installation position of the camera device. The first behavioral information corresponding to the face image is then determined based on the position of the face image at the multiple times, where the multiple times correspond to the shooting time of each image frame in the first video.

[0125] In one possible implementation, the identification module 75 is specifically used for:

[0126] Based on the position of the face image at the multiple times, determine the movement trajectory corresponding to the face image;

[0127] Determine the start and end times of the appearance of the face image in the first video;

[0128] The difference between the termination time and the start time is determined as the action duration corresponding to the face image.

[0129] The storage module 76 is used to store the plurality of face images, as well as the first line information and M second line information corresponding to each face image, into the preset database.

[0130] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the electronic device includes:

[0131] The electronic device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, memory 292, and communication interface 293 can communicate with each other via the bus 294. The communication interface 293 can be used for information transmission. The processor 291 can invoke logical instructions stored in the memory 292 to execute the methods of the above embodiments.

[0132] Furthermore, the logic instructions in the aforementioned memory 292 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0133] The memory 292, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 292, thereby implementing the methods in the above-described method embodiments.

[0134] The memory 292 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 292 may include high-speed random access memory and may also include non-volatile memory.

[0135] This application provides a non-transitory computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the foregoing embodiments.

[0136] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0138] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An object detection method, characterized in that, include: Acquire the first video captured by the camera device within a preset area; In the first video, multiple face images are identified, as well as first behavioral information corresponding to each face image. The first behavioral information includes the action trajectory and the action duration. M second behavioral information corresponding to each face image are obtained from a preset database. The preset database is used to store behavioral information corresponding to each face image that appears in the preset area within a historical time period. M is an integer. The second behavioral information includes the movement trajectory and the movement duration. Based on the first behavioral information and M second behavioral information corresponding to each face image, a target face image is determined among the plurality of face images, and the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to the preset probability. Based on the first row of information and M second row of information corresponding to each face image, the target face image is determined from the plurality of face images, including: Based on the first behavior information corresponding to each face image, at least one candidate face image is determined from the plurality of face images, and the action duration corresponding to the candidate face image is greater than or equal to a preset duration. Based on the number M of second behavioral information corresponding to each candidate face image and the action duration in each behavioral information, at least one selected face image is determined from the plurality of candidate face images. The number M of second behavioral information corresponding to the selected face image is greater than or equal to a preset number, and the action duration in each behavioral information corresponding to the selected face image is greater than or equal to a preset duration. Each selected face image, as well as the first behavioral information and M second behavioral information corresponding to each selected face image, are processed by a target model to determine the target face image from the at least one candidate face image.

2. The method according to claim 1, characterized in that, The target face image is determined by processing each selected face image, along with its corresponding first row information and M second row information, using a target model to identify the target face image from the at least one candidate face image. This process includes: The target model processes each selected face image, as well as the first action information and M second action information corresponding to each selected face image, to obtain the predicted probability corresponding to each selected face image. The predicted probability is the probability that the object corresponding to the selected face image will perform the preset action. Selected face images whose predicted probability is greater than or equal to the preset probability are identified as the target face images.

3. The method according to claim 2, characterized in that, The target model includes an image feature extraction layer, an action feature extraction layer, a fusion layer, and a classification layer; For any selected face image; the selected face image, along with its corresponding first row of information and M second row of information, are processed using a target model to obtain the predicted probability corresponding to the selected face image, including: The selected face image is processed by the image feature extraction layer to obtain face features; The behavior feature extraction layer processes the first behavior information and M second behavior information corresponding to the selected face image to obtain behavior features; The facial features and behavioral features are fused together using the fusion layer to obtain fused features; The fused features are processed by the classification layer to obtain the predicted probability.

4. The method according to any one of claims 1-3, characterized in that, In the first video, multiple facial images are identified, along with first behavioral information corresponding to each facial image. This first behavioral information includes movement trajectory and duration, including: Face recognition processing is performed on each image frame in the first video to obtain the plurality of face images; For any given face image, the position of the face image at multiple times is determined based on the position of the face image in each image frame of the first video and the installation position of the camera device. The first behavioral information corresponding to the face image is then determined based on the position of the face image at the multiple times, where the multiple times correspond to the shooting time of each image frame in the first video.

5. The method according to claim 4, characterized in that, Determining the first behavioral information corresponding to the face image based on the position of the face image at the multiple time points includes: Based on the position of the face image at the multiple times, determine the movement trajectory corresponding to the face image; Determine the start and end times of the appearance of the face image in the first video; The difference between the termination time and the start time is determined as the action duration corresponding to the face image.

6. The method according to any one of claims 1-3 and 5, characterized in that, The method further includes: The multiple face images, along with the first line information and M second line information corresponding to each face image, are stored in the preset database.

7. An object detection device, characterized in that, include: The acquisition module is used to acquire the first video captured by the camera device within a preset area; The determination module is used to determine multiple face images in the first video and first behavioral information corresponding to each face image, wherein the first behavioral information includes action trajectory and action duration; The query module is used to retrieve M second behavioral information corresponding to each face image from a preset database. The preset database is used to store behavioral information corresponding to each face image that appears in the preset area within a historical time period. M is an integer, and the second behavioral information includes the movement trajectory and the movement duration. The detection module is used to determine a target face image among the plurality of face images based on the first behavior information and M second behavior information corresponding to each face image, wherein the probability that the object corresponding to the target face image performs a preset behavior is greater than or equal to a preset probability. The detection module is specifically used for: Based on the first behavior information corresponding to each face image, at least one candidate face image is determined from the plurality of face images, and the action duration corresponding to the candidate face image is greater than or equal to a preset duration. Based on the number M of second behavioral information corresponding to each candidate face image and the action duration in each behavioral information, at least one selected face image is determined among the plurality of candidate face images. The number M of second behavioral information corresponding to the selected face image is greater than or equal to a preset number, and the action duration in each behavioral information corresponding to the selected face image is greater than or equal to a preset duration. The target face image is determined by processing each selected face image, as well as the first line information and M second line information corresponding to each selected face image, through the target model.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

Images