Image processing method and device, equipment and storage medium

Abstract

The invention discloses an image processing method and device, equipment and a storage medium. The method comprises the following steps: running at least two strategy modules in advance; the different strategy modules are used for acquiring images in parallel for processing; the operation target strategy module executes the following steps: carrying out locking processing on a to-be-processed image; the to-be-processed image corresponds to a target object detection frame determined for the to-be-processed image in advance; the target strategy module is used for processing a target object in the image; for the to-be-processed image, obtaining target image content including a target object based on a corresponding target object detection frame; and under a preset unlocking condition, performing unlocking processing on the to-be-processed image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] Currently, there are many processing strategies for image processing, such as face recognition, vehicle recognition, trajectory tracking, etc. In actual business, it is often necessary to combine multiple processing strategies to process images.

[0003] For example, for several consecutive video frame images in a video, two processing strategies, face recognition and face tracking, can be used to identify the people and their motion trajectories in the video.

[0004] In order to improve the efficiency of image processing, images can be processed in parallel according to different processing strategies.

[0005] However, when images are processed in parallel according to different processing strategies, preemption often occurs, resulting in low efficiency of image processing. Summary of the Invention

[0006] The present invention provides an image processing method, apparatus, device and storage medium to address the deficiencies in the related art.

[0007] According to a first aspect of an embodiment of the present invention, there is provided an image processing method, comprising:

[0008] At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing; and the target strategy module is run to perform the following steps:

[0009] Performing locking processing on the image to be processed; the image to be processed corresponds to a target object detection frame pre-determined for the image to be processed; the target strategy module is used to process the target object in the image;

[0010] For the image to be processed, obtaining target image content containing the target object based on the corresponding target object detection frame;

[0011] In a preset unlocking situation, unlocking processing is performed on the image to be processed.

[0012] Optionally, for the image to be processed, obtaining target image content containing the target object based on the target object detection frame includes:

[0013] Acquire image content within a preset detection frame in the image to be processed; the image content within the preset detection frame includes image content within the target object detection frame;

[0014] The acquired image content is target image content containing the target object.

[0015] Optionally, the target policy module is not used to update the image; the preset unlocking situation includes: a situation where the target policy module obtains the target image content.

[0016] Optionally, the image to be processed corresponds to a target tracking result pre-determined for the image to be processed; the target tracking result includes the target object detection frame; and the target strategy module is further configured to process the target tracking result corresponding to the image;

[0017] The steps of running the target policy module also include:

[0018] Performing locking processing on the target tracking result corresponding to the image to be processed;

[0019] Obtaining a target tracking result corresponding to the image to be processed;

[0020] In a preset unlocking situation, unlocking processing is performed based on the target tracking result corresponding to the image to be processed.

[0021] Optionally, the target strategy module is not used to update the image and the target tracking result corresponding to the image;

[0022] The preset unlocking situation includes: the target policy module obtaining the target image content and the target tracking result corresponding to the image to be processed.

[0023] Optionally, the locking process includes: acquiring a read lock; and the unlocking process includes: releasing the read lock.

[0024] Optionally, the target policy module is further used to update the target tracking result corresponding to the image; the preset unlocking situation includes: the target policy module completes updating the target tracking result corresponding to the image to be processed.

[0025] Optionally, the locking process includes: acquiring a write lock; and the unlocking process includes: releasing the write lock.

[0026] Optionally, the target tracking result includes at least one of the following items corresponding to the target object detection frame:

[0027] Tracking identification, target object appearance time, target object disappearance time, image quality value and associated object;

[0028] The target tracking result also includes: lost tracking object.

[0029] Optionally, the target object detection frame includes a face detection frame and a head detection frame;

[0030] The target strategy module is also used to associate face detection frames and head detection frames;

[0031] The steps of running the target policy module also include:

[0032] In the target tracking result corresponding to the image to be processed, an association relationship between the face detection frame and the head detection frame is determined according to an intersection-over-union ratio between the face detection frame and the head detection frame.

[0033] Optionally, the image to be processed has a timestamp; the step of running the target policy module further includes:

[0034] Identify the acquired target image content and determine the target identification mark of the target object;

[0035] In the case that the pre-built dynamic time set does not include the target identification identifier, adding the corresponding relationship built between the target identification identifier and the timestamp of the image to be processed to the dynamic time set;

[0036] In a case where the target identification identifier is included in the dynamic time set, determining whether a time difference between a timestamp corresponding to the target identification identifier and a timestamp of the image to be processed is greater than a preset time period threshold;

[0037] In a case where the determined time difference is greater than the preset time period threshold, the timestamp of the image to be processed is updated to the timestamp corresponding to the target identification in the dynamic time set.

[0038] Optionally, the target tracking result includes a tracking identifier corresponding to the target object detection frame;

[0039] The steps of running the target policy module also include:

[0040] determining an image quality for the acquired target image content, and if the determined image quality is greater than a threshold quality threshold, adding the acquired target image content to a designated image set, wherein different image contents in the designated image set correspond to the same tracking identifier;

[0041] When the number of image contents with image quality greater than the high-efficiency quality threshold in the current designated image set is greater than or equal to a preset number, subsequent processing is performed according to the current designated image set.

[0042] Optionally, the pre-running of at least two policy modules includes:

[0043] Run different policy modules based on different threads in advance; or

[0044] Different policy modules are run based on different processes in advance.

[0045] According to a second aspect of an embodiment of the present invention, another image processing method is provided, including:

[0046] At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing; and the target strategy module is run to perform the following steps:

[0047] Locking is performed on the image to be processed; the target strategy module is not used to update the image;

[0048] In a preset unlocking situation, unlocking processing is performed on the image to be processed; the preset unlocking situation includes a situation where the target policy module obtains the image to be processed.

[0049] According to a third aspect of an embodiment of the present invention, there is provided an image processing apparatus, including:

[0050] At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing;

[0051] The target policy module performs the following steps:

[0052] Performing locking processing on the image to be processed; the image to be processed corresponds to a target object detection frame pre-determined for the image to be processed; the target strategy module is used to process the target object in the image;

[0053] For the image to be processed, obtaining target image content containing the target object based on the corresponding target object detection frame;

[0054] In a preset unlocking situation, unlocking processing is performed on the image to be processed.

[0055] Optionally, a target policy module is used to perform:

[0056] Acquire image content within a preset detection frame in the image to be processed; the image content within the preset detection frame includes image content within the target object detection frame;

[0057] The acquired image content is target image content containing the target object.

[0058] Optionally, the target policy module is not used to update the image; the preset unlocking situation includes: a situation where the target policy module obtains the target image content.

[0059] Optionally, the image to be processed corresponds to a target tracking result pre-determined for the image to be processed; the target tracking result includes the target object detection frame; and the target strategy module is further configured to process the target tracking result corresponding to the image;

[0060] Optionally, the target policy module is also used to perform:

[0061] Performing locking processing on the target tracking result corresponding to the image to be processed;

[0062] Obtaining a target tracking result corresponding to the image to be processed;

[0063] In a preset unlocking situation, unlocking processing is performed based on the target tracking result corresponding to the image to be processed.

[0064] Optionally, the target strategy module is not used to update the image and the target tracking result corresponding to the image;

[0065] The preset unlocking situation includes: the target policy module obtaining the target image content and the target tracking result corresponding to the image to be processed.

[0066] Optionally, the locking process includes: acquiring a read lock; and the unlocking process includes: releasing the read lock.

[0067] Optionally, the target policy module is further used to update the target tracking result corresponding to the image; the preset unlocking situation includes: the target policy module completes updating the target tracking result corresponding to the image to be processed.

[0068] Optionally, the locking process includes: acquiring a write lock; and the unlocking process includes: releasing the write lock.

[0069] Optionally, the target tracking result includes at least one of the following items corresponding to the target object detection frame:

[0070] Tracking identification, target object appearance time, target object disappearance time, image quality value and associated object;

[0071] The target tracking result also includes: lost tracking object.

[0072] Optionally, the target object detection frame includes a face detection frame and a head detection frame;

[0073] The target strategy module is also used to associate face detection frames and head detection frames;

[0074] Optionally, the target policy module is further configured to:

[0075] In the target tracking result corresponding to the image to be processed, an association relationship between the face detection frame and the head detection frame is determined according to an intersection-over-union ratio between the face detection frame and the head detection frame.

[0076] Optionally, the image to be processed has a timestamp; the target strategy module is further configured to:

[0077] Identify the acquired target image content and determine the target identification mark of the target object;

[0078] In the case that the pre-built dynamic time set does not include the target identification identifier, adding the corresponding relationship built between the target identification identifier and the timestamp of the image to be processed to the dynamic time set;

[0079] In a case where the target identification identifier is included in the dynamic time set, determining whether a time difference between a timestamp corresponding to the target identification identifier and a timestamp of the image to be processed is greater than a preset time period threshold;

[0080] In a case where the determined time difference is greater than the preset time period threshold, the timestamp of the image to be processed is updated to the timestamp corresponding to the target identification in the dynamic time set.

[0081] Optionally, the target tracking result includes a tracking identifier corresponding to the target object detection frame;

[0082] The target policy module is also used to:

[0083] determining an image quality for the acquired target image content, and if the determined image quality is greater than a threshold quality threshold, adding the acquired target image content to a designated image set, wherein different image contents in the designated image set correspond to the same tracking identifier;

[0084] When the number of image contents with image quality greater than the high-efficiency quality threshold in the current designated image set is greater than or equal to a preset number, subsequent processing is performed according to the current designated image set.

[0085] Optionally, the pre-running of at least two policy modules includes:

[0086] Run different policy modules based on different threads in advance; or

[0087] Different policy modules are run based on different processes in advance.

[0088] According to a fourth aspect of an embodiment of the present invention, there is provided an image processing apparatus, including:

[0089] At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing;

[0090] The target policy module performs the following steps:

[0091] Locking is performed on the image to be processed; the target strategy module is not used to update the image;

[0092] In a preset unlocking situation, unlocking processing is performed on the image to be processed; the preset unlocking situation includes a situation where the target policy module obtains the image to be processed.

[0093] According to the above embodiments, by locking the image to be processed, access protection is provided for the image to be processed, and the strength of access protection is improved, thereby reducing the preemption of different policy modules, reducing the risk of image processing errors, and improving image processing efficiency.

[0094] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0096] Figure 1 is a flowchart of an image processing method according to an embodiment of the present invention;

[0097] Figure 2 is a flowchart of another image processing method according to an embodiment of the present invention;

[0098] Figure 3 1 is a schematic diagram showing a principle of video stream processing according to an embodiment of the present invention;

[0099] Figure 4 This is a schematic diagram of a process of copying information according to an embodiment of the present invention;

[0100] Figure 5 is a schematic diagram of a process for updating a dynamic time list according to an embodiment of the present invention;

[0101] Figure 6 is a schematic structural diagram of an image processing device according to an embodiment of the present invention;

[0102] Figure 7 The figure is a schematic diagram of the hardware structure of a computer device configured with the method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0103] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0104] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0105] Currently, there are many processing strategies for image processing, such as face recognition, vehicle recognition, trajectory tracking, etc. In actual business, it is often necessary to combine multiple processing strategies to process images.

[0106] For example, for several consecutive video frame images in a video, two processing strategies, face recognition and face tracking, can be used to identify the people and their motion trajectories in the video.

[0107] In order to improve the efficiency of image processing, images can be processed in parallel according to different processing strategies.

[0108] However, when images are processed in parallel according to different processing strategies, preemption often occurs, resulting in low efficiency of image processing.

[0109] The embodiment of the present invention discloses an image processing method.

[0110] In this method, considering that different processing strategies process images in parallel, preemption may occur, which leads to errors in image processing and further reduces the efficiency of image processing.

[0111] Therefore, in this method, the image to be processed can be locked. Specifically, a locking mechanism can be adopted, such as a read lock, a write lock, a mutex lock, a spin lock, etc., to provide access protection for the image to be processed, improve the strength of access protection, and thereby reduce the preemption of various processing strategies, reduce the risk of errors in image processing, and improve image processing efficiency.

[0112] Specifically, each processing strategy may lock the image to be processed before processing the image. For example, it may be a mutex lock, so that a single processing strategy can exclusively process the image to be processed, and other processing strategies cannot process the image to be processed. This reduces the situation where each processing strategy preempts the other, reduces the risk of errors in image processing, and improves image processing efficiency.

[0113] The specific locking type can be selected according to the situation and requirements of the processing strategy.

[0114] For example, for a processing strategy that only needs to read an image for processing, a read lock can be added to the image to be processed; for a processing strategy that needs to update the image, that is, a write operation needs to be performed on the image, a write lock can be added to the image to be processed.

[0115] After the processing strategy locks the image to be processed, the image can be further acquired for processing. When acquiring the image content, it can be acquired according to the requirements of the processing strategy, thereby improving image processing efficiency.

[0116] For example, for a processing strategy that only needs to process part of the image content, only the content that needs to be processed can be obtained for subsequent processing. Compared with directly obtaining the entire image content, this can improve image processing efficiency by reducing the amount of image content obtained.

[0117] Corresponding to locking the image to be processed, it is also necessary to further unlock the locked image to be processed.

[0118] This method does not limit the specific unlocking time. Specifically, the unlocking process can be performed after the processing strategy is completed, or the unlocking time can be determined according to the situation and needs of the processing strategy.

[0119] For example, for a processing strategy that only needs to read an image for processing and does not update it, you can unlock it directly after obtaining the image content, without waiting for the processing strategy to complete. Compared to unlocking after the processing strategy is completed, this can improve image processing efficiency by reducing the lock time of the image to be processed.

[0120] An image processing method provided by an embodiment of the present invention is explained in detail below.

[0121] In the process of this method, the target strategy module for processing the target object in the image is mainly explained. By obtaining part of the image content containing the target object, the amount of image content required to be obtained can be reduced, the efficiency of obtaining the image content can be improved, and the efficiency of image processing can be further improved.

[0122] like Figure 1 As shown, Figure 1FIG. 4 is a flow chart of an image processing method according to an embodiment of the present invention.

[0123] The embodiment of the present invention does not limit the execution subject of the method process. Optionally, the execution subject can be any computing device, for example, a server for image processing, or a terminal for performing image processing.

[0124] At least two strategy modules may be pre-run in the process of the method; different strategy modules may be used to acquire images in parallel for processing.

[0125] The method may include the following steps: running the target policy module to execute the following steps S101-S103.

[0126] S101: performing locking processing on the image to be processed; the image to be processed corresponds to a target object detection frame pre-determined for the image to be processed; and the target strategy module is used to process the target object in the image.

[0127] S102: For the image to be processed, obtain target image content containing the target object based on the corresponding target object detection frame.

[0128] S103: Under the preset unlocking condition, unlocking processing is performed on the image to be processed.

[0129] The above method process can provide access protection for the image to be processed by locking the image to be processed, thereby improving the strength of access protection, thereby reducing the preemption of different strategy modules, reducing the risk of image processing errors, and improving image processing efficiency.

[0130] In addition, the above method flow can also be used for the target strategy module for processing the target object in the image. By pre-determining the target object detection frame for the image to be processed, the image content containing the target object can be obtained based on the target object detection frame for subsequent processing. Compared with directly obtaining the entire image content of the image to be processed, the image processing efficiency can be improved by reducing the amount of image content obtained.

[0131] Optionally, the target policy module may be any policy module. For the convenience of description, any policy module targeted by the process of this method is referred to as the target policy module.

[0132] The above method flow explains the process of running the target policy module to lock and unlock, and the process of other policy modules obtaining images for processing can also refer to the above method flow and related explanations.

[0133] Regarding the policy module, the method flow does not limit the specific form of the policy module.

[0134] Optionally, the policy module may specifically be a program code for processing an image, a virtual device for processing an image, or software or a component for processing an image.

[0135] This method does not limit the specific way of processing images in the policy module.

[0136] Optionally, the policy module can be used to recognize faces in an image, determine the number of people in an image, perform target detection on an image, track targets on an image, determine image quality, and the like.

[0137] This method does not limit the relationship between policy modules.

[0138] Optionally, different policy modules may perform different and unrelated processing on an image. For example, one policy module may perform face detection on an image, while another policy module may perform vehicle recognition on an image.

[0139] Optionally, there may be a sequential processing relationship between different policy modules. For example, one policy module may perform face detection on an image, and another policy module may determine the number of detected faces based on the face detection result.

[0140] It should be noted that different strategy modules can be used to acquire images for processing in parallel. Regardless of whether the different strategy modules perform unrelated processing or have a sequential processing relationship, images can be acquired for processing in parallel to improve image processing efficiency.

[0141] For example, two policy modules that process images sequentially are referred to as Policy Module 1 and Policy Module 2 for ease of description. Policy Module 2 requires the processing results of Policy Module 1. Therefore, Policy Module 2 can retrieve images from images already processed by Policy Module 1 for processing, or it can perform secondary processing on the image results already retrieved by Policy Module 1. The image retrieval operations can be performed in parallel between Policy Module 1 and Policy Module 2. Of course, Policy Module 2 can also directly retrieve images for processing, then cease processing upon determining that the image has not been processed by Policy Module 1.

[0142] For easier understanding, a specific example is given below.

[0143] In a specific example, a business party needs to determine traffic flow and identify license plate numbers of vehicles based on a video of vehicle traffic.

[0144] Therefore, the business party can deploy at least three policy modules. Policy module one can be used to identify vehicles in video frames, policy module two can be used to count the number of vehicles, and policy module three can be used to determine the license plate number of the vehicle.

[0145] The three policy modules can obtain video frames from the video stream in parallel for processing. Specifically, the policy module one, the policy module two, and the policy module three can obtain video frames in parallel for processing respectively.

[0146] Specifically, the first strategy module obtains the video frame to identify the vehicle, the second strategy module obtains the video frame of the identified vehicle to count, and the third strategy module obtains the video frame of the identified vehicle and further identifies the license plate number. Of course, the third strategy module can also directly obtain the video frame to identify the license plate number.

[0147] The method does not limit the specific way of running the policy module. Optionally, the policy module can be run in a thread or process.

[0148] Optionally, at least two policy modules are pre-run, specifically: different policy modules are pre-run based on different threads respectively; or different policy modules are pre-run based on different processes respectively.

[0149] By running different policy modules in different threads or processes, images can be acquired and processed in parallel, improving image processing efficiency. To reduce preemption between threads or processes, a lock can be added before acquiring the image. Specifically, a read lock or a write lock can be added, depending on actual needs.

[0150] Optionally, the target policy module is run, specifically, the target policy module is run based on a thread or a process.

[0151] The process of this method does not limit the form and content of the image to be processed. Optionally, the image to be processed can be any image, specifically any image that the target policy module needs to process.

[0152] In a specific example, the image to be processed may be a video frame, specifically any video frame in a video stream; the image to be processed may also be any image in an image set.

[0153] Different strategy modules can be used to obtain images from a video stream or image collection in parallel for processing.

[0154] The following is a detailed explanation of each step.

[0155] 1. S101: Locking processing is performed on the image to be processed; the image to be processed corresponds to the target object detection frame pre-determined for the image to be processed; the target strategy module is used to process the target object in the image.

[0156] Regarding the image to be processed, the method flow is not limited to a specific form and determination method.

[0157] Optionally, any image may be determined as the image to be processed, any image may be determined from an image set as the image to be processed, or any video frame may be determined from a video as the image to be processed. Of course, an image may also be specified as the image to be processed, specifically by selecting an image from an image set as the image to be processed.

[0158] The locking process is performed on the image to be processed. Optionally, after the image to be processed is determined, the locking process is further performed on the image to be processed to determine whether the locking is successful.

[0159] For ease of understanding, in a specific example, if the determined image to be processed is previously write-locked by another policy module, the target policy module cannot add a write lock or a read lock to the image to be processed.

[0160] Of course, after determining the image to be processed, it may be determined whether the image to be processed can be locked. If it is determined that the image to be processed can be locked, the image to be processed may be locked.

[0161] For example, if the determined image to be processed is not currently locked by other policy modules, the image to be processed can be locked; or if the image to be processed is currently read locked or shared locked by other policy modules, the image to be processed can be read locked.

[0162] The process of this method does not limit the specific locking processing method. Optionally, any of the following locks can be added: mutex lock, spin lock, shared lock, read lock and write lock.

[0163] For example, a read lock or a write lock of the image to be processed may be acquired to implement locking processing.

[0164] It should be noted that, in order to reduce the situation of preemption, the image can be locked before it is acquired. After the lock is successfully acquired, the image content can be further acquired.

[0165] Therefore, optionally, any image in the image set may be determined as the image to be processed, and a lock process may be performed on the image to be processed. If the lock process is successful, subsequent steps may be performed. Specifically, the image set may be a video stream.

[0166] Regarding the target object, the present method flow does not limit the specific form and content of the target object.

[0167] Optionally, the target object may be at least one of the following: a vehicle, a face, a head, a body, a license plate, an object, a building, and the like.

[0168] In the process of this method, the target strategy module can be used to process the target object in the image. For example, the target strategy module can be used to perform identity recognition on a face, or can be used to perform number recognition on a license plate, etc.

[0169] In the process of this method, a target object detection frame in the image to be processed can be predetermined.

[0170] This method does not limit the method for determining the target object detection frame in the image. Specifically, target detection can be used to pre-determine the target object detection frame within the image. The specific target detection algorithm is not limited, and examples include neural network target detection and graph convolutional network target detection.

[0171] In a specific example, the target object may be a face, and face detection may be performed on the image in advance to determine a face detection frame therein. The target object may also be a vehicle, and vehicle detection may be performed on the image in advance to determine a vehicle detection frame therein.

[0172] Of course, since the target objects are not limited in type and number, the target objects may include faces, heads and vehicles, so that the face detection frame, head detection frame and vehicle detection frame in the image can be determined respectively based on face detection, head detection and vehicle detection.

[0173] The process of this method uses a predetermined target object detection frame to facilitate the subsequent steps of the target strategy module for the target object in the image that needs to be processed, and directly obtains part of the image content based on the target object detection frame. Compared with directly obtaining the entire image to be processed, the amount of image content required to be obtained can be reduced, the efficiency of obtaining image content can be improved, and thus the efficiency of image processing can be improved.

[0174] 2. S102: For the image to be processed, obtain target image content containing the target object based on the corresponding target object detection frame.

[0175] The method flow does not limit the execution order of the steps between S101 and S102.

[0176] In an optional embodiment, S102 may be executed after S101. Optionally, specifically, when the locking process is successful, that is, when the locking is successful, S102 is executed to further obtain the image content of the image to be processed.

[0177] In this embodiment, it is possible to first determine that the lock is successful and then perform image processing, which can reduce the situation of preemption.

[0178] In the case of locking failure, the present method does not limit the specific operation. Optionally, S101 can be re-executed to obtain other images or repeat the locking process until the locking is successful, or S102 can be directly executed.

[0179] Of course, S101 and S102 can also be performed in parallel. While the image to be processed is being locked, the image content of the image to be processed is obtained. Optionally, if the locking is successful, the subsequent steps can be continued. If the locking fails, the subsequent steps can be stopped, or the obtained image content can be deleted.

[0180] The process of this method is not limited to a specific method of obtaining the target image content based on the target object detection frame, as long as the image content containing the target object can be obtained.

[0181] Optionally, the image content to be processed in the target object detection frame can be directly obtained as the target image content; or the target object detection frame can be enlarged and then the image content to be processed in the enlarged detection frame can be obtained, so as to facilitate the acquisition of more image content near the target object, thereby improving the accuracy of image processing.

[0182] The size of the target image content may be smaller than the image to be processed, thereby reducing the amount of image content that needs to be acquired, improving the efficiency of image content acquisition, and improving the efficiency of image processing.

[0183] The specific method of obtaining the image content is not limited in the present method flow, and can be specifically copying the image content.

[0184] Optionally, for the image to be processed, target image content containing the target object is obtained based on the target object detection frame. Specifically, image content within the target object detection frame in the image to be processed is obtained as the target image content containing the target object.

[0185] Optionally, for the image to be processed, target image content containing the target object is obtained based on the target object detection frame. Specifically, the following steps may be performed: obtaining image content within a preset detection frame in the image to be processed; the image content within the preset detection frame may include image content within the target object detection frame; and the obtained image content may be target image content containing the target object. The size of the preset detection frame may be larger than or equal to the target object detection frame. The predicted detection frame may include the target object detection frame.

[0186] It should be noted that the predicted detection frame can correspond one-to-one with a single target object detection frame. For multiple target object detection frames in the image to be processed, the image content of each target object detection frame can be obtained separately. Specifically, the image content of each preset detection frame can be obtained separately. The preset detection frames correspond one-to-one with the target object detection frames, and the image content of the preset detection frames can include the image content of the corresponding target object detection frame.

[0187] This embodiment can obtain image content within a preset detection frame to facilitate the acquisition of more image content near the target object, thereby improving the accuracy of image processing.

[0188] In an optional embodiment, the target object detection frame corresponding to the image to be processed may be filtered first, and then the image content within the filtered target object detection frame is obtained.

[0189] This embodiment does not limit the specific screening method, which can be set according to the actual business or policy module requirements.

[0190] For example, a strategy module for processing faces requires that the resolution of the face image content be greater than the preset resolution to improve the accuracy of subsequent face processing. The filtering condition can be set to be that the resolution of the target object detection box is greater than the preset resolution.

[0191] Therefore, optionally, for the image to be processed, the target image content containing the target object is obtained based on the target object detection frame. Specifically, it can be: for the image to be processed, the target image content containing the target object is obtained based on the target object detection frame that meets the preset filtering conditions.

[0192] Of course, further screening can also be performed after obtaining the image content.

[0193] This embodiment does not limit specific preset filtering conditions. Optionally, the preset filtering conditions may include at least one of the following: the size of the target object detection frame is larger than a preset size, the area of ​​the target object detection frame is larger than a preset area, the resolution of the target object detection frame is larger than a preset resolution, the confidence level of the target object detection frame is greater than a preset confidence level, etc.

[0194] In an optional embodiment, since it is necessary to obtain the image content in combination with the target object detection frame, the target object detection frame may also be locked.

[0195] The steps of running the target policy module may also include: locking the target object detection frame corresponding to the image to be processed; obtaining the target object detection frame corresponding to the image to be processed; and unlocking the target object detection frame corresponding to the image to be processed under a preset unlocking condition.

[0196] After obtaining the target object detection frame, the target image content including the target object can be further obtained based on the corresponding target object detection frame for the image to be processed.

[0197] The preset unlocking situation here may specifically include a situation where a target object detection frame corresponding to the image to be processed is obtained, and may also be combined with a situation where the image to be processed is unlocked.

[0198] By locking the target object detection frame, access protection can be provided for the target object detection frame, reducing the preemption of various policy modules, reducing the risk of image processing errors, and improving image processing efficiency.

[0199] For detailed explanations of the locking and unlocking processes, please refer to the explanations of other embodiments.

[0200] The type of lock applied to the image to be processed can be the same as or different from the type of lock applied to the target object detection frame. For example, both can be read locks or both can be write locks. Alternatively, a write lock can be applied to the image to be processed and a read lock can be applied to the target object detection frame.

[0201] 3. S103: Under the preset unlocking condition, unlocking processing is performed on the image to be processed.

[0202] This method process is not limited to a specific preset unlocking situation.

[0203] Optionally, the preset unlocking conditions can be determined based on the specific needs of the target policy module. For example, after the target policy module completes processing, the image to be processed can be unlocked; or after the target policy module updates the image to be processed, the image to be processed can be unlocked, specifically by performing operations such as sharpening or increasing the resolution of the image to be processed.

[0204] For example, after the target policy module obtains the target image content, the remaining steps are for the target policy module to process it by itself. There is no need to read or update the image to be processed, and the image to be processed can be unlocked. This method can also speed up the unlocking efficiency, reduce the time the image to be processed is locked, improve the efficiency of other policy modules in locking the image to be processed, and improve the efficiency of the overall image processing.

[0205] For more specific explanations of preset unlocking situations, please see below.

[0206] This method does not limit the specific unlocking process. The specific unlocking process can correspond to the previous locking process. For example, if the previous locking process was to add a read lock, the unlocking process can be to release the read lock. If the previous locking process was to add a write lock, the unlocking process can be to release the write lock.

[0207] More specifically, the previous locking process may be acquiring a read lock of the image to be processed, and the unlocking process may be releasing the read lock of the image to be processed. The previous locking process may be acquiring a write lock of the image to be processed, and the unlocking process may be releasing the write lock of the image to be processed.

[0208] The following is a detailed explanation of the preset unlocking situation.

[0209] In an optional embodiment, the corresponding preset unlocking situation may be determined based on whether the target policy module updates the image.

[0210] First of all, it should be noted that the target strategy module can be used to obtain images for processing. Of course, the explanation of other strategy modules can refer to the explanation of the target strategy module.

[0211] The target strategy module may process the image in any manner, for example, identifying a target object in the image, determining the number of target objects in the image, or updating the image, specifically by improving image quality, changing image resolution, and the like.

[0212] The updated image can be further acquired and processed by other policy modules after being unlocked.

[0213] Optionally, the target policy module can determine an image to be processed from the image collection, lock the image to be processed, and then use the target policy module to update the image to be processed in the image collection. Specifically, the target policy module can update the image to be processed in the image collection, specifically, updating the entire image to be processed in the image collection, or a portion of the image to be processed in the image collection. That is, updating the entire content or a portion of the content of the image to be processed in the image collection. The updated image to be processed in the image collection can continue to be determined by the policy module.

[0214] For ease of understanding, in a specific example, the target policy module can be specifically used to improve the image resolution so that the image to be processed can be obtained. After the image resolution is improved, the updated image to be processed is added to the image collection, making it convenient for other policy modules to obtain the image with improved resolution for processing.

[0215] Of course, the target strategy module may not update the image. For example, the target strategy module may be used to identify the number of target objects in the image, specifically the number of vehicles. In other words, the target strategy module may not update the image, but may calculate the number of vehicles after obtaining image content containing vehicles in the image to be processed.

[0216] Optionally, for target policy modules that don't update images, to increase unlocking speed and reduce the duration of locked images, the image to be processed can be unlocked after the target policy module obtains the target image content. Since the target policy module processes the target image content after obtaining it, it does not affect the images to be processed in the image collection. Therefore, the image can be unlocked and the image content can be obtained by other policy modules.

[0217] Therefore, optionally, the target policy module is not used to update the image; the preset unlocking situation may specifically be: the target policy module obtains the target image content.

[0218] For target policy modules that do not update images, the specific locking process can use a read lock. Since the read lock is a shared lock, it can facilitate other policy modules to also read the image to be processed. Correspondingly, the unlocking process can be to release the read lock, specifically, to release the read lock of the image to be processed.

[0219] Optionally, the locking process may be performed by acquiring a read lock, that is, acquiring a read lock of the image to be processed; and the unlocking process may be performed by releasing a read lock, that is, releasing the read lock of the image to be processed.

[0220] This embodiment can increase the unlocking speed, reduce the time it takes to lock the image to be processed, and improve the overall image processing efficiency.

[0221] Optionally, for the target policy module that updates the image, the image to be processed can be unlocked after the image update is completed. This embodiment does not limit the specific updated image portion, and the image can be the entire image or part of it.

[0222] Therefore, optionally, the target policy module may be used to update the image; the preset unlocking situation may specifically be: the situation where the target policy module completes the image update.

[0223] Corresponding to the target policy module that needs to update the image, in order to reduce preemption, the specific locking processing can use a write lock or other mutex lock to monopolize the image to be processed. After the image update is completed, it can be unlocked, specifically by unlocking the write lock or releasing the mutex lock of the image to be processed.

[0224] 4. About target tracking.

[0225] In an optional embodiment, different strategy modules may be used to obtain images from an image collection in parallel for processing.

[0226] The images in an image set may be associated with each other. For example, the image set may include consecutive video frames or consecutively captured images. Accordingly, to facilitate the use of subsequent strategy modules, target tracking can be performed on the image set to obtain target tracking results.

[0227] In one example, the image collection can specifically include consecutive video frame images from a video. Different policy modules can be used to obtain the video frame images from the video for processing. Target tracking can be performed on the video in advance to obtain target tracking results. Examples include face tracking and vehicle tracking. Each video frame image in the video can correspond to its own target tracking result, specifically including the detected target object and the tracking identifier assigned to the target object, facilitating the determination of the trajectory of target objects with the same tracking identifier across video frames.

[0228] Therefore, optionally, the image to be processed may correspond to a target tracking result determined in advance for the image to be processed. This embodiment does not limit the specific method of determining the target tracking result, and a target tracking algorithm may be used.

[0229] This embodiment does not limit the form of the target tracking result. Optionally, it can be in the form of a tracking list, a tracking set, etc.

[0230] This embodiment also does not limit the content of the target tracking result, and specifically does not limit the target tracking result corresponding to the image to be processed.

[0231] In an optional embodiment, target tracking may include performing target detection on the current image and then matching the detected target object in the previous image to determine the association between the target object detected in the current image and the target object in the previous image. Specifically, the matching may be performed using the intersection-over-union ratio between the target object detection frames.

[0232] It is understandable that the target tracking result may include the target object detection frame explained in the above embodiment.

[0233] For the target object detection frame, the tracking identifier of the target object detection frame can be determined through matching association, and the appearance time and disappearance time of the target object corresponding to the tracking identifier can be determined. The appearance time can be the time when the target object is first detected, and the disappearance time can be the time when the target object is no longer detected.

[0234] In addition, the target tracking results can also include quality values ​​obtained by performing quality assessment on the entire image or a part of the image. For the target object detection frame, the relationship between other target object detection frames in the same image can also be determined, which can generally be the relationship between different types of target objects.

[0235] For example, for the face detection frame and head detection frame in the same image, association matching can be performed to determine that the face and head belong to the same person, which facilitates subsequent processing.

[0236] Therefore, optionally, the target tracking result may include at least one of the following items corresponding to the target object detection frame: a tracking identifier, a target object appearance time, a target object disappearance time, an image quality value, and an associated object.

[0237] In an optional embodiment, the detected target object may be a tracking object, so that multiple attributes of the tracking object can be determined. These tracking objects can be stored in a tracking object list to facilitate display of specific attributes.

[0238] In contrast to the tracking objects, the disappeared tracking objects can be first determined as lost tracking objects. Specifically, the lost tracking objects can be stored in a lost tracking list to facilitate the display of specific attributes.

[0239] Specifically, determining that the tracking object is lost may be determining that the tracking object does not appear again in several consecutive frames, so that the tracking object is considered to have disappeared and is determined to be lost.

[0240] Retaining the lost tracking objects can facilitate subsequent determination of whether the lost tracking objects appear again, can facilitate determination of the number of lost tracking objects, and can also facilitate some policy modules to process the lost tracking objects.

[0241] Optionally, the target tracking result may further include: lost tracking object.

[0242] In an optional embodiment, the target strategy module may also be used to process the target tracking result corresponding to the image.

[0243] This embodiment does not limit the specific method of processing the target tracking results. Optionally, the target tracking results corresponding to the image can be used to filter out tracked objects for analyzing object trajectories, or to filter out tracked objects for identifying the objects, or to filter out tracked objects with low confidence, etc.

[0244] In a specific example, the strategy module can be used to comprehensively determine the trajectory of any target object, such as the trajectory of any vehicle, based on the target tracking results.

[0245] Since the target strategy module needs to process the target tracking results, access protection can be provided through a locking mechanism.

[0246] Optionally, the steps executed by running the target strategy module may also include: locking the target tracking result corresponding to the image to be processed; obtaining the target tracking result corresponding to the image to be processed; and unlocking the target tracking result corresponding to the image to be processed under a preset unlocking condition.

[0247] After obtaining the target tracking result, the target tracking result can be further processed.

[0248] The preset unlocking situation here may specifically include a situation where a target tracking result corresponding to the image to be processed is obtained, and may also be combined with a situation where the image to be processed is unlocked.

[0249] By locking the target tracking results, access protection can be provided for the target tracking results, reducing the preemption of various strategy modules, reducing the risk of image processing errors, and improving image processing efficiency.

[0250] For detailed explanations of the locking and unlocking processes, please refer to the explanations of other embodiments.

[0251] The type of lock applied to the image to be processed can be the same as or different from the type of lock applied to the target tracking results. For example, both can be read locks or both can be write locks. Alternatively, a write lock can be applied to the image to be processed and a read lock can be applied to the target tracking results.

[0252] The target strategy module specifically processes the target tracking results in two ways: updating the target tracking results and not updating the target tracking results.

[0253] Corresponding to different methods of processing target tracking results, different preset unlocking conditions can be set, and different locking processing methods can also be set.

[0254] Optionally, the target policy module may not be used to update the target tracking result corresponding to the image. In other words, the target policy module may read the target tracking result corresponding to the image without updating it. After the target policy module obtains the target tracking result corresponding to the image, it may unlock the target tracking result.

[0255] Therefore, optionally, when the target policy module obtains the target tracking result corresponding to the image, unlocking processing may be performed on the target tracking result corresponding to the image.

[0256] Correspondingly, a read lock or a shared lock can be used to lock the target tracking result corresponding to the image. Optionally, the target tracking result corresponding to the image to be processed can be locked, specifically by adding a read lock to the target tracking result corresponding to the image to be processed, that is, obtaining the read lock of the target tracking result corresponding to the image to be processed. The target tracking result corresponding to the image to be processed can be unlocked, specifically by releasing the read lock of the target tracking result corresponding to the image to be processed, that is, releasing the read lock of the target tracking result corresponding to the image to be processed. Similarly, the explanation of the shared lock can be found in this embodiment.

[0257] The preset unlocking situation may include a situation where the target strategy module obtains a target tracking result corresponding to the image to be processed.

[0258] Optionally, the target strategy module may also be used to update the target tracking result corresponding to the image. After completing the update of the target tracking result corresponding to the image, the target strategy module may unlock the target tracking result.

[0259] Therefore, optionally, when the target policy module completes updating the target tracking result corresponding to the image, unlocking processing may be performed on the target tracking result corresponding to the image.

[0260] Correspondingly, a write lock or a mutex lock can be used to lock the target tracking result corresponding to the image. Optionally, the target tracking result corresponding to the image to be processed can be locked, specifically by adding a write lock to the target tracking result corresponding to the image to be processed, that is, obtaining the write lock of the target tracking result corresponding to the image to be processed. The target tracking result corresponding to the image to be processed can be unlocked, specifically by releasing the write lock of the target tracking result corresponding to the image to be processed, that is, releasing the write lock of the target tracking result corresponding to the image to be processed. Similarly, the explanation of the mutex lock can be found in this embodiment.

[0261] The preset unlocking situation may include a situation where the target strategy module completes updating the target tracking result corresponding to the image to be processed.

[0262] In an optional embodiment, the unlocking of the target tracking result and the unlocking of the image to be processed may be combined.

[0263] It is understandable that the target tracking result and the locking processing method and the preset unlocking status of the image to be processed need to be determined according to the actual processing status of the target strategy module.

[0264] If the target strategy module does not update the image to be processed, nor does it update the target tracking result corresponding to the image to be processed, the locking process can specifically acquire the read lock, and the unlocking process can specifically release the read lock. The preset unlocking situation can also be the situation where the target image content of the image to be processed is obtained, and the target tracking result corresponding to the image to be processed is obtained.

[0265] Optionally, the target policy module is not used to update the image and the target tracking result corresponding to the image; the preset unlocking conditions may include: the target policy module obtains the target image content and the target tracking result corresponding to the image to be processed. The locking process may specifically include: obtaining a read lock, that is, obtaining a read lock for the image to be processed, and obtaining a read lock for the target tracking result corresponding to the image to be processed; the unlocking process may specifically include: releasing a read lock, that is, releasing the read lock for the image to be processed, and releasing the read lock for the target tracking result corresponding to the image to be processed.

[0266] If the target policy module does not update the image to be processed, but updates the target tracking result corresponding to the image to be processed, a read lock can be applied to the image to be processed and a write lock can be applied to the target tracking result corresponding to the image to be processed. Of course, both locks can also be applied. The preset unlocking condition can also be when the target image content of the image to be processed is obtained and the target tracking result corresponding to the image to be processed is updated.

[0267] If the target policy module updates the image to be processed but does not update the target tracking result corresponding to the image to be processed, a write lock can be applied to the image to be processed and a read lock can be applied to the target tracking result corresponding to the image to be processed. Of course, both write locks can also be applied. The preset unlocking condition can also be when the target image content of the image to be processed is updated and the target tracking result corresponding to the image to be processed is obtained.

[0268] If the target strategy module updates both the image to be processed and the target tracking result corresponding to the image to be processed, the locking process can specifically acquire the write lock, and the unlocking process can specifically release the write lock. The preset unlocking situation can also be the situation where the target image content of the image to be processed is updated and the target tracking result corresponding to the image to be processed is updated.

[0269] Therefore, it can be determined that when the target policy module updates the image to be processed or the target tracking result corresponding to the image to be processed, access protection can be provided by adding a write lock and a preset unlocking condition can be set accordingly.

[0270] Optionally, the target policy module may also be used to update the target tracking result corresponding to the image; the preset unlocking situation may specifically include: the target policy module completes updating the target tracking result corresponding to the image to be processed.

[0271] Optionally, the target policy module can also be used to update the image and the target tracking result corresponding to the image; the preset unlocking situation can specifically include: the target policy module completes updating the target image content of the image to be processed, and completes updating the target tracking result corresponding to the image to be processed.

[0272] Among them, the locking process can be specifically: obtaining a write lock, that is, obtaining a write lock of the image to be processed, and obtaining a write lock of the target tracking result corresponding to the image to be processed; the unlocking process can be specifically: releasing a write lock, that is, releasing the write lock of the image to be processed, and releasing the write lock of the target tracking result corresponding to the image to be processed.

[0273] It should be noted that regarding the locking process, the images to be processed and the target tracking results can be locked separately or simultaneously. Regarding the preset unlocking conditions, the images to be processed and the target tracking results can be unlocked separately and in parallel under specified circumstances, or simultaneously under the preset unlocking conditions.

[0274] For example, in parallel, when the target policy module completes updating the target image content of the image to be processed, the image to be processed is unlocked; when the target policy module completes updating the target tracking result corresponding to the image to be processed, the target tracking result corresponding to the image to be processed is unlocked.

[0275] 5. Association between head and face.

[0276] For ease of understanding, in an optional embodiment, the target object may be limited to include a human head and a human face.

[0277] Heads and faces are often associated, meaning they belong to the same person. Therefore, by associating detected heads and faces, we can facilitate target tracking. This is because faces in a video may move and become undetectable, while associated heads can still be detected. Therefore, we can determine whether the associated face has lost tracking based on whether the associated head has been detected.

[0278] If the associated head is also lost from the video, it can be determined that the associated face has lost tracking. If the associated head is still detected in the video, even if the associated face is temporarily not detected, perhaps because the head is turned or lowered, it can be determined that the associated face has not lost tracking. This can reduce the risk of face loss or false detection.

[0279] Optionally, the target object detection frame may include a face detection frame and a head detection frame; the target strategy module may also be used to associate the face detection frame and the head detection frame; the steps of running the target strategy module may also include: in the target tracking result corresponding to the image to be processed, according to the intersection-and-union ratio between the face detection frame and the head detection frame, determining the association relationship between the face detection frame and the head detection frame.

[0280] Among them, the target strategy module can be used to associate the face detection frame and the head detection frame in the image to be processed, and specifically can be used to determine the face detection frame and the head detection frame belonging to the same person.

[0281] This embodiment does not limit the specific method of determining the association relationship based on the intersection-over-union ratio.

[0282] Optionally, for each head detection frame, the face detection frame with the largest intersection-over-union ratio can be determined as the associated face detection frame; or for each face detection frame, the head detection frame with the largest intersection-over-union ratio can be determined as the associated head detection frame.

[0283] Of course, the size and resolution of the head detection frame and the face detection frame can also be considered. Optionally, the head detection frame and the face detection frame can be sorted from largest to smallest according to resolution. Then, the head detection frames can be traversed in descending order of resolution, and the face detection frame with the highest intersection-union ratio can be determined with the head detection frames in descending order of resolution. After the associated face detection frame is determined, the determined face detection frame can be deleted from the sorting.

[0284] When specifically associating the face detection frame and the head detection frame, the face identifier and the head identifier can be used for association.

[0285] In a specific example, the movement trajectory of a person can be identified based on the detected and tracked head and face, and the person's identity can be identified based on the face. The identified person's identity can be used to determine whether a warning message should be triggered. Specifically, a warning message can be triggered if a person violating the rules is identified.

[0286] During the tracking process, the face is easily lost due to reasons such as turning or lowering the head. Therefore, the head can be further tracked by associating it with the head, reducing the risk of losing tracking and improving the integrity of the person's movement trajectory.

[0287] 6. Object recognition and dynamic temporal aggregation.

[0288] In an optional embodiment, the target policy module can also identify a target object in the image. For example, if the target object is a vehicle, the target policy module can be used to identify the vehicle's identification, specifically the license plate number. For another example, if the target object is a face, the target policy module can be used to identify the person's identification, specifically the person's information.

[0289] This embodiment does not limit the specific object recognition method. Optionally, face recognition, vehicle recognition, license plate recognition, etc. can be used.

[0290] During the object recognition process of the target policy module, corresponding processing can generally be performed based on the recognition results. For example, based on the recognized person identification, it is determined whether a warning message needs to be triggered; based on the recognized vehicle identification, it is determined whether a notification message needs to be triggered, etc.

[0291] Taking into account the situation of lost tracking, some target objects may be re-detected within a short period of time. For example, when a person is at the edge of the image, sometimes it cannot be detected and sometimes it can be detected; or people often enter and exit the picture, resulting in sometimes successful detection and sometimes failed detection within a short period of time.

[0292] Correspondingly, if the corresponding processing operation is directly triggered for such a re-detected situation, redundant and repeated operations will often occur.

[0293] For example, within a continuous 10-second image, a violator can be detected and tracked for the first 2 seconds, but not for the middle 3 seconds. Then, he can be detected and tracked again for the last 3 seconds, but loses tracking again for the last 2 seconds. This will trigger two repeated alarms within 10 seconds.

[0294] In order to save processing operations and improve processing efficiency, this embodiment can combine time and recognition results to construct an information set.

[0295] This information set can store the identified object identifiers and the corresponding identification times. Furthermore, when the same object identifier is identified, it can be determined whether the duration between the corresponding identification times is greater than a preset duration. If it is less than the preset duration, it can be determined that redundant processing operations may occur, and the corresponding operation can be triggered without triggering the corresponding operation. If it is greater than the preset duration, the corresponding operation can be triggered directly.

[0296] Therefore, optionally, the image to be processed may have a timestamp; the step of running the target policy module may also include: identifying the acquired target image content to determine the target identification identifier of the target object; when the pre-constructed dynamic time set does not contain the target identification identifier, adding the correspondence between the target identification identifier and the timestamp of the image to be processed to the dynamic time set; when the dynamic time set contains the target identification identifier, determining whether the time difference between the timestamp corresponding to the target identification identifier and the timestamp of the image to be processed is greater than a preset time period threshold; when the determined time difference is greater than the preset time period threshold, updating the timestamp of the image to be processed to the timestamp corresponding to the target identification identifier in the dynamic time set.

[0297] This embodiment does not limit the specific size of the preset time period threshold. Optionally, it can be set to 10 seconds or 10 minutes.

[0298] In order to conveniently control the size of the dynamic time set, optionally, the corresponding relationship where the time difference between the corresponding timestamp and the current time is greater than a preset clearing period threshold can be deleted.

[0299] This embodiment can update and maintain the dynamic time set, so as to trigger corresponding processing operations according to the identification identifier in the dynamic time set.

[0300] In the case that the determined time difference is less than or equal to the preset time period threshold, no operation may be performed, the dynamic time set may not be updated, and no corresponding processing operation may be triggered.

[0301] This embodiment does not limit the method for determining the timestamp of the image to be processed. Alternatively, the image to be processed may be a video frame in a video, and thus may have a video playback timestamp. The image to be processed may also be an image in a sequence of continuously captured images, and thus may have a capture timestamp.

[0302] 7. Quality screening.

[0303] In an optional embodiment, target objects with the same tracking identifier can be obtained through target tracking, thereby facilitating subsequent processing.

[0304] For example, determining the trajectory of a target object can also be used to comprehensively identify the target object, etc.

[0305] In a specific example, the face detection frames of the same face in multiple images can be integrated to determine the movement trajectory of the face, and the identification of the face can also be comprehensively identified to improve the accuracy of face recognition.

[0306] The image to be processed here can be a video frame in a video or an image in a continuously shot image sequence, so as to facilitate target tracking.

[0307] To facilitate subsequent processing, image quality can be further determined, and high-quality image content can be selected for subsequent processing. Whether determining trajectory or object recognition, high-quality image content can improve processing efficiency and accuracy.

[0308] Therefore, the image quality can be determined for the acquired target image content, and then the image content with image quality greater than a preset quality value can be screened out for subsequent processing.

[0309] In order to improve processing efficiency, the selected high-quality image contents may be stored in a set, which may be specifically referred to as a designated image set.

[0310] Furthermore, as high-quality image content accumulates, the needs for subsequent processing can be better met, so that after accumulating high-quality image content to a certain extent, there is no need to collect image content or determine the quality of image content, which reduces operations and improves efficiency.

[0311] This embodiment does not limit the specific requirement of accumulating high-quality image content.

[0312] Optionally, a quantity requirement for high-quality image content may be set, or both a quality requirement and a quantity requirement for high-quality image content may be set, and so on.

[0313] For ease of description, in this embodiment, the lower quality threshold for screening out image content is referred to as the threshold quality threshold, and the higher quality threshold for determining high-quality image content is referred to as the high-efficiency quality threshold. The high-efficiency quality threshold may be greater than the threshold quality threshold.

[0314] Optionally, the target tracking result may include a tracking identifier corresponding to the target object detection box; the step of running the target strategy module may also include: determining the image quality for the acquired target image content, and when the determined image quality is greater than a threshold quality threshold, adding the acquired target image content to a specified image set, wherein different image contents in the specified image set correspond to the same tracking identifier; when the number of image contents in the current specified image set whose image quality is greater than the high-efficiency quality threshold is greater than or equal to a preset number, performing subsequent processing according to the current specified image set.

[0315] In this embodiment, if the current image in the specified image set meets the preset requirements, that is, the amount of high-quality image content is greater than the preset amount, the high-quality image content is the image content whose image quality is greater than the high-efficiency quality threshold. Therefore, the image content no longer needs to be collected into the specified image set, and the image quality no longer needs to be determined. Subsequent processing can be performed directly to improve processing efficiency.

[0316] Another image processing method provided by an embodiment of the present invention is explained in detail below.

[0317] In the process of this method, the target policy module that is not used to update the image is mainly explained. By unlocking the image after obtaining the image content, the efficiency of image unlocking can be improved, the length of time the image is locked can be reduced, and the efficiency of the overall image processing can be improved.

[0318] like Figure 2 As shown, Figure 2 FIG. 4 is a flow chart of another image processing method according to an embodiment of the present invention.

[0319] The embodiment of the present invention does not limit the execution subject of the method process. Optionally, the execution subject can be any computing device, for example, a server for image processing, or a terminal for performing image processing.

[0320] At least two strategy modules may be pre-run in the process of the method; different strategy modules may be used to acquire images in parallel for processing.

[0321] The method may include the following steps: running the target policy module to execute the following steps S201-S202.

[0322] S201: Locking the image to be processed; the target policy module is not used to update the image.

[0323] S202: Under a preset unlocking condition, unlocking processing is performed on the image to be processed; the preset unlocking condition includes a condition in which the target policy module obtains the image to be processed.

[0324] The above method process can provide access protection for the image to be processed by locking the image to be processed, thereby improving the strength of access protection, thereby reducing the preemption of different strategy modules, reducing the risk of image processing errors, and improving image processing efficiency.

[0325] In addition, the above method process can also be used for a target policy module that is not used to update the image. When the target policy module obtains the image to be processed, the image to be processed is unlocked, which can improve the efficiency of unlocking the image to be processed, reduce the time the image to be processed is locked, and improve the overall image processing efficiency.

[0326] In an optional embodiment, the image to be processed may correspond to a target object detection frame pre-determined for the image to be processed, and the target policy module may be further configured to process the target object in the image. Optionally, the step of executing the target policy module may further include, for the image to be processed, obtaining target image content containing the target object based on the corresponding target object detection frame. The preset unlocking condition may also include: the target policy module obtaining the target image content.

[0327] The process of this method does not limit the specific way of locking. Optionally, a read lock, a write lock, or a mutual exclusion lock can be added.

[0328] For a detailed explanation of the process of this method, please refer to the explanation of S101-S103.

[0329] For easier understanding, the embodiment of the present invention further provides a specific application example.

[0330] Typically, in surveillance scenarios, a video stream needs to be pulled and decoded by a processor, and the decoded image frame is distributed to different strategies for different processing.

[0331] In this embodiment, for multiple strategies, independent multi-thread processing is adopted, and access protection is provided by adding thread locks.

[0332] The currently used frameworks are Figure 3 As shown, Figure 3 The figure is a schematic diagram showing the principle of video stream processing according to an embodiment of the present invention.

[0333] The decoded image is then passed through a unified detection model (multi-branch detection model) to obtain different detection results simultaneously. Currently, face detection, head detection, human body detection, and vehicle detection are commonly used. The detection results are tracked and processed according to the different labels.

[0334] When multiple strategies process the same video stream, they are generally processed independently by multiple threads and require thread locks to prevent the threads from interfering with each other.

[0335] If there are three strategies, then these three strategies are processed simultaneously by multiple threads. Each strategy takes the required tracking results. For example, the face-related strategy only needs to take the face tracking results. The tracking results include: the current tracking list and the lost tracking list. The tracking list includes: the tracking box (face detection box) and the tracking ID as well as the first appearance time, disappearance time and other information.

[0336] After each strategy obtains the current frame image and tracking results, it needs to add a thread lock to process the strategy process. At this time, other strategies cannot obtain the current frame image and tracking results until the current frame is processed.

[0337] Take the tracking list as an example. In the first frame, 4 targets are detected. At this time, there will be 4 in the current tracking list and the lost tracking list will be empty.

[0338] If these four targets can be detected before the nmth frame, then the tracking IDs of these four targets are always 1, 2, 3, and 4.

[0339] When these four targets disappear and cannot be detected in the n-m+1th frame, then these four targets will not appear in the current tracking list, nor will they appear in the lost tracking list. At this time, there is a temporary storage life cycle, which is set according to the specific situation. It is only to avoid assigning multiple tracking IDs to the same target. If the temporary storage life cycle is 5, then if these four targets are not detected for 5 consecutive frames, then these four targets will appear in the lost tracking list in the n-m+5th frame.

[0340] 1. In order to improve image processing efficiency, this embodiment proposes the following solution.

[0341] For strategies that only need to process tracking results, after each strategy locks the current frame image and tracking results, the tracking results corresponding to the current frame image can be copied in the read lock, the thread lock can be unlocked, and then the copied tracking results can be processed according to the strategy process.

[0342] Compared to the solution of waiting until the processing is completed before unlocking, this can reduce the time the strategy takes to process the image and tracking results without affecting the use of other strategies.

[0343] For strategies that need to process the target object in the image, each strategy locks the current frame image and tracking results, then copies the partial image content containing the target object, unlocking the thread lock. The copied partial image content is then processed.

[0344] Compared with the solution of copying the entire image content, this can reduce the time the strategy takes to process the image and tracking results without affecting the use of other strategies.

[0345] Of course, for strategies that need to process the entire image, the entire image content still needs to be copied.

[0346] For strategies that need to process target objects and tracking results in images, after locking the current frame image and tracking results, each strategy can copy the tracking results corresponding to the current frame image, and the copied image contains the local image content of the target object to unlock the thread lock.

[0347] This can reduce the time the strategy takes to process images and tracking results without affecting the use of other strategies.

[0348] Specifically, the partial image content may be copied by expanding the width and height of the face detection frame along the center point of the detection frame by N times (usually N is between 1 and 2 according to the requirements, generally N=1.1), and then copying the image content in the expanded detection frame.

[0349] Copying the enlarged face image is almost time-free, which not only solves the time-consuming problem of copying the entire image content, but also improves the processing frame rate of the strategy in the overall framework.

[0350] For easier understanding, this embodiment also provides a specific process.

[0351] That is, after first copying the tracking information from the current tracking list during the read lock, the face bbox size is used to determine whether the current face image needs to be cut out. This is because the face size must be determined in the subsequent process. Faces smaller than the set threshold will not have their key points acquired. Only after acquiring the key points can the face be aligned, and only after the alignment, the face quality can be achieved. Therefore, here we first determine whether the face bbox size is smaller than the set threshold. Only faces larger than the set threshold will have their face images cut out and saved. In the lost tracking list, only the tracking information is lost during the copy.

[0352] The specific processing flow chart is as follows Figure 4 As shown, Figure 4 The figure is a schematic diagram of a process of copying information according to an embodiment of the present invention.

[0353] Specifically, each face detection frame in the current tracking list may be traversed first, and the corresponding current tracking result may be copied. The tracking result may include a tracking identifier.

[0354] It can then be determined whether the traversed face detection frame is smaller than a preset face size threshold.

[0355] When the traversed face detection frame is smaller than the preset face size threshold, the current tracking list can continue to be traversed.

[0356] When the traversed face detection frame is not smaller than a preset face size threshold, the image content in the enlarged face detection frame may be copied and the current tracking list may be continued to be traversed.

[0357] After traversing the current tracking list, you can continue to traverse the lost tracking list and copy the tracking information in the lost tracking list one by one.

[0358] Next, we need to align the face based on the obtained key points and the original image according to the principle of affine transformation, that is, straighten the face. The aligned face is used for face quality assessment and subsequent face recognition.

[0359] 2. Association between head and face.

[0360] In this embodiment, a head detection frame may also be identified, and then based on the intersection-over-union ratio between the face detection frame and the head detection frame, it may be determined that the head detection frame and the face detection frame belong to the same person.

[0361] During the tracking process, faces can easily lose track due to rotation, while head tracking is relatively more stable. Subsequent processing can be performed based on the head tracking results.

[0362] 3. Add dynamic time list:

[0363] Face recognition: Usually, the features of the face to be recognized are pre-extracted and stored in a database. Different databases are distinguished according to different situations, such as whitelist, blacklist, VIP, etc. The facial features stored in the database can be ID photos or clear faces captured in surveillance scenes.

[0364] Even if the head-face association is added, the loss of tracking due to head occlusion may cause one person to trigger multiple recognition messages during face recognition. Therefore, it is proposed to add a dynamic time list of recognition IDs.

[0365] That is, when someone is identified as blacklisted person (face recognition database) ID1 at time t1, the message is reported to the backend and the dynamic time list is updated.

[0366] If this ID does not exist in the dynamic time list, add it directly to the list<ID1,t1> If there is an ID corresponding to t0, it is determined whether the time interval between the two times is greater than the set time threshold time_thr.

[0367] If the time is greater than the set threshold, it is considered that the person disappeared after the last alarm and reappeared. The time interval between the two alarms is long, and the alarm should be issued again to update the dynamic time list.<ID1,t0> Updated to<ID1,t1> .

[0368] If it is less than the set time threshold, it is considered that the time interval between the two alarms is short and no alarm is required again, and the dynamic time list is not updated.

[0369] If tracking is lost due to occlusion of the face or head, and then detected and tracked again, a new tracking ID will be generated. If the same identification ID1 is recognized again, the time is t2, and it is compared with the time t1 of the last message sent in the dynamic time list.

[0370] If the time interval between two messages is too short, (t2-t1) is less than the set time threshold time_thr, the message will not be sent at this time. Otherwise, the dynamic time list will be updated.<ID1,t1> Updated to<ID1,t2> .

[0371] The reason why it is called a dynamic time list is that this list will not grow indefinitely. Instead, each time a new ID pair is added to the list, the earliest time of entry in the list is determined. If the earliest time of entry is greater than the time interval threshold time_interval_thr compared to the current time, the earliest ID pair is deleted. That is, the dynamic time list will not grow indefinitely and cause memory leaks.

[0372] The specific process is as follows Figure 5 shown. Figure 5 FIG. 4 is a flow chart showing a dynamic time list update process according to an embodiment of the present invention.

[0373] The facial identifier ID2 may be identified first, and the corresponding time t4 may be determined.

[0374] Then, the duration between the earliest time in the current dynamic time list and the current time is determined, and it is judged whether the determined duration is greater than the time interval threshold.

[0375] When the determined duration is not greater than the time interval threshold, the corresponding relationship to which the earliest time belongs is deleted, and it is determined whether the identified identifier ID2 exists in the previous dynamic time list.

[0376] When the determined duration is greater than the time interval threshold, it is further determined whether the identified identifier ID2 exists in the current dynamic time list.

[0377] If the identified ID2 does not exist in the current dynamic time list, the corresponding relationship can be added to the current dynamic time list.<ID2,t4> , that is, the identified identifier ID2 and the corresponding time t4.

[0378] When the identified identifier ID2 exists in the current dynamic time list, it is determined whether the duration between the corresponding times t5 and t4 is greater than the time threshold.

[0379] If the duration between t5 and t4 is greater than the time threshold, a corresponding relationship can be added to the current dynamic time list.<ID2,t4> , that is, the identified ID2 and the corresponding time t4.<ID2,t5> Updated to<ID2,t4> .

[0380] In the case that the duration between t5 and t4 is not greater than the time threshold, no operation may be performed.

[0381] Then you can end this process.

[0382] 4. Keep multiple high-quality faces.

[0383] Face-related strategies are generally divided into real-time processing and delayed processing. Real-time processing means performing recognition as long as certain conditions are met during face tracking, while delayed processing means sending recognition results after the face is lost. Usually, the highest-quality face image during the tracking process is used for attribute recognition or ID recognition.

[0384] Here, we can use the N images with the highest quality, that is, retain the N images with the highest quality during the face tracking process, which is equivalent to each face going through the steps of face detection → face key points → face alignment → face quality.

[0385] However, in order to save computing resources, this embodiment selects a face image with the lowest quality in the quality list. When the face image is greater than the set quality threshold quality_thr, there is no need to update the quality list. That is, the subsequent faces with the current tracking ID do not need to go through the steps of face key points → face alignment → face quality. Faces with a quality greater than quality_thr can basically obtain the correct attributes and ID.

[0386] For example, the quality list of a person's face is quality1>quality2>quality3...>

[0387] After qualityn, qualityn>quality_thr, we no longer need to perform the steps of face key points → face alignment → face quality. If qualityn is less than quality_thr, we still need to find a face with a value greater than qualityn to replace qualityn.

[0388] To increase the robustness of recognition results, we don't rely solely on the results of a single image. For attribute recognition, we vote on the N identified results to get the final result. For ID recognition, we extract N features from these N images, add and normalize the features, and then compare the resulting feature result with the feature library. This way, we only need to identify once to get a single result, increasing recognition robustness.

[0389] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of the various technical features in the above embodiments also falls within the scope of the present invention.

[0390] Corresponding to the above method embodiments, the embodiments of the present invention also provide corresponding device embodiments.

[0391] like Figure 6 As shown, Figure 61 is a schematic diagram showing the structure of an image processing device according to an embodiment of the present invention. At least two strategy modules can be pre-run in the device; different strategy modules are used to acquire images in parallel for processing.

[0392] The device may include a target strategy module 301, which is used to perform the following steps: locking the image to be processed; the image to be processed corresponds to a target object detection frame pre-determined for the image to be processed; the target strategy module is used to process the target object in the image; for the image to be processed, the target image content containing the target object is obtained based on the corresponding target object detection frame; in a preset unlocking condition, unlocking the image to be processed.

[0393] Optionally, the target policy module 301 is configured to execute:

[0394] Obtaining the image content within a preset detection frame in the image to be processed; the image content within the preset detection frame includes the image content within the target object detection frame;

[0395] The acquired image content is target image content containing the target object.

[0396] Optionally, the target policy module 301 is not used to update the image; the preset unlocking situation includes: the situation where the target policy module 301 obtains the target image content.

[0397] Optionally, the image to be processed corresponds to a target tracking result pre-determined for the image to be processed; the target tracking result includes a target object detection frame; the target strategy module 301 is further configured to process the target tracking result corresponding to the image;

[0398] Optionally, the target policy module 301 is further configured to execute:

[0399] Lock the target tracking result corresponding to the image to be processed;

[0400] Obtain the target tracking result corresponding to the image to be processed;

[0401] In the preset unlocking situation, the unlocking process is performed based on the target tracking result corresponding to the image to be processed.

[0402] Optionally, the target strategy module 301 is not used to update the image and the target tracking result corresponding to the image;

[0403] The preset unlocking situation includes: the target strategy module 301 obtains the target image content and the target tracking result corresponding to the image to be processed.

[0404] Optionally, performing locking processing includes: acquiring a read lock; performing unlocking processing includes: releasing the read lock.

[0405] Optionally, the target strategy module 301 is further used to update the target tracking result corresponding to the image; the preset unlocking situation includes: the target strategy module 301 completes updating the target tracking result corresponding to the image to be processed.

[0406] Optionally, performing locking processing includes: acquiring a write lock; performing unlocking processing includes: releasing the write lock.

[0407] Optionally, the target tracking result includes at least one of the following items corresponding to the target object detection frame:

[0408] Tracking identification, target object appearance time, target object disappearance time, image quality value and associated object;

[0409] The target tracking results also include: lost tracking objects.

[0410] Optionally, the target object detection frame includes a face detection frame and a head detection frame;

[0411] The target strategy module 301 is also used to associate the face detection frame and the head detection frame;

[0412] Optionally, the target policy module 301 is further configured to:

[0413] In the target tracking result corresponding to the image to be processed, the association relationship between the face detection frame and the head detection frame is determined according to the intersection-over-union ratio between the face detection frame and the head detection frame.

[0414] Optionally, the image to be processed has a timestamp; the target strategy module 301 is further configured to:

[0415] Identify the acquired target image content and determine the target identification mark of the target object;

[0416] In the case that the pre-built dynamic time set does not contain the target identification identifier, the corresponding relationship built between the target identification identifier and the timestamp of the image to be processed is added to the dynamic time set;

[0417] In the case where the dynamic time set includes a target identification identifier, determining whether the time difference between the timestamp corresponding to the target identification identifier and the timestamp of the image to be processed is greater than a preset time period threshold;

[0418] When the determined time difference is greater than a preset time period threshold, the timestamp of the image to be processed is updated to the timestamp corresponding to the target identification in the dynamic time set.

[0419] Optionally, the target tracking result includes a tracking identifier corresponding to the target object detection frame;

[0420] The target policy module 301 is further configured to:

[0421] determining an image quality for the acquired target image content, and if the determined image quality is greater than a threshold quality threshold, adding the acquired target image content to a designated image set, wherein different image contents in the designated image set correspond to the same tracking identifier;

[0422] When the number of image contents with image quality greater than the high-efficiency quality threshold in the current designated image set is greater than or equal to a preset number, subsequent processing is performed according to the current designated image set.

[0423] Optionally, at least two policy modules are pre-run, including:

[0424] Run different policy modules based on different threads in advance; or

[0425] Different policy modules are run based on different processes in advance.

[0426] For detailed explanation, please refer to the above method embodiment.

[0427] An embodiment of the present invention also provides an image processing device, which pre-runs at least two policy modules; different policy modules are used to obtain images in parallel for processing; the device may include a target policy module, which is used to perform the following steps: locking the image to be processed; the target policy module is not used to update the image; in a preset unlocking situation, unlocking the image to be processed; the preset unlocking situation includes the situation where the target policy module obtains the image to be processed.

[0428] For detailed explanation, please refer to the above method embodiment.

[0429] An embodiment of the present invention further provides a computer device, which includes at least a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above method embodiments is implemented.

[0430] An embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned method embodiments.

[0431] Figure 71 is a schematic diagram illustrating the hardware structure of a computer device configured with a method according to an embodiment of the present invention. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other within the device via the bus 1050.

[0432] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0433] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present invention are implemented through software or firmware, the relevant program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0434] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0435] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0436] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0437] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the embodiments of the present invention, and does not necessarily include all the components shown in the figure.

[0438] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements any of the above method embodiments when executed by a processor.

[0439] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program implements any of the above method embodiments when executed by a processor.

[0440] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0441] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the embodiments of the present invention can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the embodiments of the present invention can essentially or in other words, the contributing part can be embodied in the form of a software product. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0442] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0443] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and when implementing the embodiment of the present invention, the functions of each module can be implemented in the same one or more software and / or hardware. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0444] The above description is only a specific implementation of the embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present invention. These improvements and modifications should also be regarded as protection for the embodiment of the present invention.

[0445] In the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more, unless otherwise clearly defined.

[0446] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0447] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing; the method includes: To run the target policy module, perform the following steps: Performing locking processing on the image to be processed; the image to be processed corresponds to a target object detection frame pre-determined for the image to be processed; the target strategy module is used to process the target object in the image; For the image to be processed, obtaining target image content containing the target object based on the corresponding target object detection frame; In a preset unlocking situation, unlocking processing is performed on the image to be processed.

2. The method according to claim 1, characterized in that The step of obtaining target image content containing a target object based on a target object detection frame for the image to be processed includes: Acquire image content within a preset detection frame in the image to be processed; the image content within the preset detection frame includes image content within the target object detection frame; The acquired image content is target image content containing the target object.

3. The method according to claim 1, characterized in that The target policy module is not used to update the image; the preset unlocking situation includes: the situation where the target policy module obtains the target image content.

4. The method according to claim 1, wherein The image to be processed corresponds to a target tracking result determined in advance for the image to be processed; the target tracking result includes the target object detection frame; The target strategy module is also used to process the target tracking result corresponding to the image; The steps of running the target policy module also include: Performing locking processing on the target tracking result corresponding to the image to be processed; Obtaining a target tracking result corresponding to the image to be processed; In a preset unlocking situation, unlocking processing is performed based on the target tracking result corresponding to the image to be processed.

5. The method according to claim 4, characterized in that The target strategy module is not used to update the image and the target tracking result corresponding to the image; The preset unlocking situation includes: the target policy module obtaining the target image content and the target tracking result corresponding to the image to be processed.

6. The method according to claim 3 or 5, characterized in that The locking process includes: acquiring a read lock; the unlocking process includes: releasing the read lock.

7. The method according to claim 4, characterized in that The target strategy module is further used to update the target tracking result corresponding to the image; the preset unlocking situation includes: the target strategy module completes updating the target tracking result corresponding to the image to be processed.

8. The method according to claim 7, characterized in that The locking process includes: acquiring a write lock; the unlocking process includes: releasing the write lock.

9. The method according to claim 4, characterized in that The target tracking result includes at least one of the following items corresponding to the target object detection frame: Tracking identification, target object appearance time, target object disappearance time, image quality value and associated object; The target tracking result also includes: lost tracking object.

10. The method according to claim 4, characterized in that The target object detection frame includes a face detection frame and a head detection frame; The target strategy module is also used to associate face detection frames and head detection frames; The steps of running the target policy module also include: In the target tracking result corresponding to the image to be processed, an association relationship between the face detection frame and the head detection frame is determined according to an intersection-over-union ratio between the face detection frame and the head detection frame.

11. The method according to claim 1, wherein The image to be processed has a timestamp; The steps of running the target policy module also include: Identify the acquired target image content and determine the target identification mark of the target object; In the case that the pre-built dynamic time set does not include the target identification identifier, adding the corresponding relationship built between the target identification identifier and the timestamp of the image to be processed to the dynamic time set; In a case where the target identification identifier is included in the dynamic time set, determining whether a time difference between a timestamp corresponding to the target identification identifier and a timestamp of the image to be processed is greater than a preset time period threshold; In a case where the determined time difference is greater than the preset time period threshold, the timestamp of the image to be processed is updated to the timestamp corresponding to the target identification in the dynamic time set.

12. The method according to claim 4, characterized in that The target tracking result includes a tracking identifier corresponding to the target object detection frame; The steps of running the target policy module also include: determining an image quality for the acquired target image content, and if the determined image quality is greater than a threshold quality threshold, adding the acquired target image content to a designated image set, wherein different image contents in the designated image set correspond to the same tracking identifier; When the number of image contents with image quality greater than the high-efficiency quality threshold in the current designated image set is greater than or equal to a preset number, subsequent processing is performed according to the current designated image set.

13. The method according to claim 1, wherein The pre-running of at least two policy modules includes: Run different policy modules based on different threads in advance; or Different policy modules are run based on different processes in advance.

14. An image processing method, characterized in that: At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing; the method includes: To run the target policy module, perform the following steps: Locking is performed on the image to be processed; the target strategy module is not used to update the image; In a preset unlocking situation, unlocking processing is performed on the image to be processed; the preset unlocking situation includes a situation where the target policy module obtains the image to be processed.

15. An image processing device, characterized in that: At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing; the device comprises: The target policy module performs the following steps: Performing locking processing on the image to be processed; the image to be processed corresponds to a target object detection frame pre-determined for the image to be processed; the target strategy module is used to process the target object in the image; For the image to be processed, obtaining target image content containing the target object based on the corresponding target object detection frame; In a preset unlocking situation, unlocking processing is performed on the image to be processed.

16. An image processing device, characterized in that: At least two strategy modules are pre-run; different strategy modules are used to acquire images in parallel for processing; the device comprises: The target policy module performs the following steps: Locking is performed on the image to be processed; the target strategy module is not used to update the image; In a preset unlocking situation, unlocking processing is performed on the image to be processed; the preset unlocking situation includes a situation where the target policy module obtains the image to be processed.

17. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 14.

18. A computer-readable storage medium storing a computer program, characterized in that: The computer program implements the method according to any one of claims 1 to 14 when executed by a processor.

Images