Target analysis method and electronic device

Abstract

The present application discloses a target analysis method and electronic device, including: inputting image data to be processed into at least two different types of target analysis modules; each of the target analysis modules processes the image data respectively to obtain respective target analysis results; each of the target analysis results includes at least one target and its trajectory information, and at least one target analysis result includes the structured information of the target; matching the trajectory information of the targets in the target analysis results obtained by different target analysis modules, determining the structured information of the first target corresponding to the successfully matched trajectory information in each target analysis result; and outputting the result based on the structured information of the first target. By using the present application, it is possible to integrate a variety of different types of target analysis results to meet complex analysis requirements.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to a target analysis method and electronic equipment. Background Art

[0002] With the advancement of network technology and image processing technology, video surveillance systems have been increasingly widely used.

[0003] At present, various target detection and target analysis can be achieved through surveillance video. For example, indoor target detection can monitor the indoor environment. After detecting the target that needs to be alarmed in the indoor environment, the target trajectory can be obtained through target tracking and alarm information can be generated; structured target detection can detect the target, obtain the target trajectory through target tracking, and obtain the structured information of the target. Among them, structured information usually refers to various attributes of the target, including but not limited to the height, body size, movement direction, limb length, head length, pupil distance, target posture, etc. detected for people, and the color, type, shape, etc. of the vehicle detected for the vehicle. Target analysis is usually performed on dynamic targets (mainly refers to targets that are not completely stationary or targets that are not without behavioral changes). Therefore, the target structured information involved in target analysis is usually attribute information for dynamic targets.

[0004] However, the current multiple target analysis and processing are independent of each other, and the target analysis and processing results are also processed or output separately. Since each target analysis and processing is carried out from different angles, the analysis and processing from different angles are independent of each other and cannot meet some complex analysis requirements. Summary of the invention

[0005] The present application provides a target analysis method and electronic device that can integrate a variety of different types of target analysis results to meet complex analysis needs.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] A target analysis method, comprising:

[0008] Inputting the image data to be processed into at least two different types of target analysis modules;

[0009] Each of the target analysis modules processes the image data respectively to obtain respective target analysis results; wherein each of the target analysis results includes at least one target and its trajectory information, and at least one target analysis result includes structural information of the target;

[0010] Matching the trajectory information of the targets in the target analysis results obtained by the different types of target analysis modules, and determining the first target corresponding to the successfully matched trajectory information in the corresponding target analysis results;

[0011] Based on the structured information of the first target, the result is output.

[0012] Preferably, the target analysis module is a target detection alarm unit and a structured target detection unit; wherein the target detection alarm unit is used to detect an alarm event and an alarm target associated with the alarm event, and output alarm information indicating the occurrence of the alarm event and the alarm target and its trajectory information, and the structured target detection unit is used to perform structured information detection of the target, and output the structured target and its structured information and trajectory information;

[0013] The matching of trajectory information of targets in target analysis results obtained by different types of target analysis modules includes:

[0014] For each alarm target output by the target detection alarm unit, the trajectory information of the alarm target is matched with the trajectory information of each structured target output by the structured target detection unit; wherein the structured target is the target in the target analysis result output by the structured target detection unit.

[0015] Preferably, the method of determining whether the match is successful includes:

[0016] Based on the time information in the image data, among the trajectory information of all structured objects, the trajectory information of the structured objects that matches the trajectory of the alarm object in the same time period is determined as the trajectory information that has been successfully matched.

[0017] Preferably, the outputting of results based on the structured information of the first target includes:

[0018] Performing secondary analysis based on the structured information of the first target, and outputting results based on the secondary analysis results;

[0019] Alternatively, the structured information of the first target is directly output as a result.

[0020] Preferably, the performing secondary analysis based on the structured information of the first target includes:

[0021] Based on the structured information of the first target output by the structured target detection unit, performing target modeling of the output first target;

[0022] Based on the result of the target modeling, searching in the target model library to obtain the target search result of the first target, and saving the result of the target modeling in the target model library;

[0023] Each alarm information output by the target detection alarm unit and the associated alarm target and the target retrieval result of the first target are correspondingly output.

[0024] Preferably, there are multiple target detection alarm units, and the alarm events of different target detection alarm units are different;

[0025] Matching the alarm target trajectory information output by each of the indoor target detection alarm units with the target trajectory information output by the structured target detection unit to determine a first target that matches successfully;

[0026] The method further comprises:

[0027] The target retrieval results of the first target corresponding to different target detection alarm units are matched and compared, and the best matching target retrieval result and its corresponding alarm information are output.

[0028] Preferably, the target detection alarm unit is an indoor target detection alarm unit, the determined alarm information is a close contact alarm, the alarm targets corresponding to the alarm information are two detected targets meeting the close contact condition, and the target modeling is face modeling;

[0029] When the trajectory information of the alarm target is matched with the trajectory information of each of the structured targets respectively, the matching is performed for each of the two targets; the alarm information is associated with the two alarm targets;

[0030] For each of the two output structured targets, perform the face modeling and the retrieval to obtain face retrieval results for each of the two output structured targets;

[0031] The face retrieval results of each alarm information and its corresponding two output structured targets are output accordingly.

[0032] Preferably, the step of determining the trajectory information of the structured target having the highest matching degree with the trajectory of the alarm target in the same time period as the successfully matched trajectory information includes:

[0033] The trajectory information of the alarm target is compared with the trajectory information of each structured target respectively. Based on the curve fitting effect of the trajectory, the degree of overlap of the trajectory positions in the entire frame image, and the degree of consistency of the timestamp information corresponding to the trajectory points at the same position, the trajectory of the structured target that matches the trajectory of the alarm target is selected from the trajectories of the various structured targets.

[0034] Preferably, when the target detection alarm unit does not output alarm information, the method further comprises:

[0035] Selecting a structured object from all structured objects output by the structured object detection unit;

[0036] Based on the structured information of the selected structured target, target modeling is performed, and the result of the target modeling is stored in the target model library.

[0037] An electronic device comprising at least a computer-readable storage medium and a processor;

[0038] The processor is used to read executable instructions from the computer-readable storage medium and execute the instructions to implement any of the target analysis methods described above.

[0039] As can be seen from the above technical solution, in the present application, first, the image data to be processed is input into at least two different types of target analysis modules, and each target analysis module is used to process the image data respectively to obtain respective target analysis results. In this way, the processing of image data can be realized by different types of target analysis modules, so as to obtain target analysis results of different angles of the same image data. Among them, each target analysis result includes at least one target and its trajectory information, and at least one target analysis result includes the structured information of the target. Next, the targets in the target analysis results obtained by each target analysis module are matched with trajectory information, and the first target corresponding to the successfully matched trajectory information in each target analysis module is determined. In this way, the analysis results of different angles can be linked together through trajectory matching, the target analysis results obtained from different angles are integrated, and the successfully matched target is determined. Finally, based on the structured information of the first target that successfully matched, the result of the target analysis is output. Thus, the integration of multiple different types of target analysis results based on trajectory information can be realized to meet more complex analysis needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the basic process of the target analysis method in this application;

[0041] Figure 2 This is a schematic diagram of a specific process of the target analysis method in a specific embodiment of the present application;

[0042] Figure 3 It is a schematic diagram of the alarm target trajectory of the perimeter detection alarm unit in the application example;

[0043] Figure 4 A schematic diagram of the detection target trajectory of the structured target detection unit in the application example;

[0044] Figure 5 It is a schematic diagram of the basic structure of the target analysis device in this application;

[0045] Figure 6 It is a schematic diagram of the basic structure of the electronic device in this application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical means and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings.

[0047] Figure 1 The following is a basic flow chart of the target analysis method in this application. Figure 1 As shown, the method includes:

[0048] Step 101: input the image data to be processed into at least two different types of target analysis modules.

[0049] The image data to be processed may be a single frame of image data or a group of image frame data consisting of multiple image frames, such as a video or animation. At least two different types of target analysis modules are introduced in the present application to perform different types of target analysis processing on the same image data to be processed. Thus, the same group of image data is input into multiple target analysis modules of different types.

[0050] Step 102: Each target analysis module processes the image data respectively to obtain respective target analysis results.

[0051] Each target analysis result includes at least one target and its trajectory information, and at least one target analysis result includes structured information of the target.

[0052] Specifically, each target analysis module processes the input image data independently of each other, and the internal processing of the specific target analysis module can be performed according to the existing processing method, and the corresponding target analysis result is obtained. In general, in this step, for a single target analysis module, the image data is processed and output according to the existing method to obtain the corresponding target analysis result.

[0053] Step 103 , matching the trajectory information of the targets in the target analysis results obtained by the different types of target analysis modules, and determining the first target corresponding to the successfully matched trajectory information in the corresponding target analysis results.

[0054] In the present application, for the target analysis results of different types of target analysis modules obtained in step 102, the targets therein are subjected to trajectory information matching processing. For the successfully matched trajectory information, the corresponding target in the corresponding target analysis result is determined, and the target is called the first target to distinguish it from other targets in the target analysis result.

[0055] Step 104: output the result of target analysis based on the structured information of the first target.

[0056] After obtaining the first target, the structured information of the first target may be directly output, or the target analysis result may be output after performing the next level of processing based on the structured information of the first target.

[0057] At this point, the target analysis method process in this application ends. Through the above process, different types of target analysis are performed on the image data, and the different target analysis results obtained are matched according to the trajectory information, the first target that matches successfully is determined, and finally the result is output based on the structured information of the first target. In this way, target analysis processing from multiple different angles can be integrated, and different target analysis results can be integrated from the perspective of trajectory information, so as to obtain target analysis results that are successfully matched by trajectory information for output or subsequent processing, thereby being able to meet more complex target analysis needs.

[0058] For complex target analysis needs, it is usually possible to match and integrate different target analysis results based on the specific analysis needs, application scenarios, etc., and perform subsequent processing based on specific needs. For example, a commonly used complex demand can be to detect and alarm targets through monitoring images, and obtain detailed information of the alarm targets. However, for ordinary target analysis and processing, the target analysis module generally used to achieve target detection and alarm can usually only achieve simple target detection, and cannot obtain detailed information of the target, while the target analysis module used to obtain detailed information of the target can be, for example, a structured target detection module, which can usually only perform target detection and obtain corresponding structured information, and cannot perform alarms according to demand. Therefore, the use of any single target analysis and processing cannot meet the needs of performing target alarms and obtaining detailed information of the alarm target. For this type of complex analysis needs, the target analysis method of the present application can be used to achieve it.

[0059] Specifically, using the method of the present application, in order to obtain the alarm target and detailed information of the target, a target detection alarm unit that can realize target detection alarm and a structured target detection unit that can realize structured detection of the target can be selected as two different types of target analysis modules. The target detection alarm unit can be used to detect alarm events and alarm targets associated with alarm events, and output alarm information indicating the occurrence of alarm events and the alarm target and its trajectory information; the structured target detection unit can be used to perform structured information detection of the target, and output the target and its structured information and trajectory information.

[0060] In more detail, a group of image frames (such as a video or a gif) can be used as the image data to be processed, and the corresponding image data can be input into the target detection alarm unit and the structured target detection unit respectively. Among them, the target detection alarm unit can detect and track one or more targets in the image data, and determine that an alarm event is detected and generate alarm information when the target meets the alarm condition. The output target analysis result includes the alarm information and the target associated with the alarm information (hereinafter referred to as the alarm target) and its trajectory; the structured target detection unit can detect and track the target in the image data, and obtain the structured information of the detected target, such as height, posture, head information, etc. The output target analysis result includes each detected target (hereinafter referred to as the structured target) as well as the target trajectory and the target structured information. In order to achieve the demand of obtaining the alarm target and its detailed structured information, the target analysis result of the target detection alarm unit can be matched with the target analysis result of the structured target detection unit according to the trajectory. The specific matching method may include:

[0061] For the alarm target corresponding to each alarm information output by the target detection alarm unit, the trajectory of the alarm target is matched with the trajectory of each structured target output by the structured target detection unit respectively; among the trajectories of all structured targets, the trajectory of the structured target that best matches the trajectory of the alarm target in the same time period is determined, and the trajectory of the determined structured target is taken as the trajectory of the successful match, that is, if the determined structured target is considered to be consistent with the alarm target, then the structured target can be taken as the structured target corresponding to the alarm information. To distinguish it from other structured targets, the structured target corresponding to the alarm information is referred to as the output structured target hereinafter; since the output structured target is considered to be consistent with the alarm target corresponding to the alarm information, the structured information of the output structured target is the detailed information of the alarm target, that is, the target detailed information that is actually desired. Based on this, next, the structured information of the successfully matched structured target can be directly output or the structured information can be used for subsequent processing before output. For example, target modeling can be performed based on the structured information, and the target model library can be searched based on the result of the target modeling to obtain the target retrieval result of the output structured target. Each alarm information output by the target detection alarm unit and its corresponding target retrieval result of the output structured target are correspondingly output, so that the alarm information and its associated target information can be more intuitively displayed. In addition, in order to improve the accuracy of subsequent target retrieval, the result of the target modeling of the structured target (i.e., the target model obtained by the target modeling) can also be output and saved in the target model library. The target modeling here can be the modeling of the entire or partial components of the structured target, for example, face modeling, etc.

[0062] In addition, the target detection alarm unit may generate multiple alarm information, each alarm information is associated with at least one alarm target, and for each alarm information and its associated alarm target, the structured information of the alarm target can be obtained in the above manner; and the target retrieval result of the alarm target can be determined and output corresponding to the alarm information. Multiple alarm information can obtain multiple corresponding target retrieval results, and each target retrieval result can be output corresponding to the corresponding alarm information.

[0063] The target analysis method in this application is described below through specific examples. Figure 2 The following is a specific flow chart of the target analysis method in the specific embodiment of this application. In this embodiment, the specific analysis requirements are: to perform indoor personnel close contact detection alarm, and determine the corresponding alarm target for face retrieval, that is, target modeling is specifically face modeling, and target retrieval is specifically face retrieval. Figure 2 As shown, the method includes:

[0064] Step 201: Input a surveillance video into an indoor target detection alarm unit and a structured target detection unit respectively.

[0065] In this embodiment, a surveillance video is used for target analysis. This surveillance video is input into two different types of target analysis modules, namely, an indoor target detection alarm unit and a structured target detection unit. Among them, the indoor target detection alarm unit is a common target detection alarm unit, which can identify various types of targets in the screen through the surveillance video, and generate alarm information when a target that meets the alarm condition appears. For example, the alarm condition can usually be to alarm when a person is recognized.

[0066] Step 202: the indoor target detection alarm unit processes the input surveillance video, generates alarm information after detecting the alarm target, and outputs the alarm information and the trajectory of the alarm target to the post-processing module.

[0067] A common indoor target detection alarm unit can detect and track targets in surveillance videos, distinguish different targets appearing in surveillance videos, and track each target one by one to obtain the target's moving trajectory. According to the pre-set alarm conditions, an alarm message is generated when a target that meets the alarm conditions is found, and the alarm target and the trajectory of the alarm target are given. For example, the alarm condition can be that a person is detected in the picture.

[0068] In this embodiment, on the basis of the above-mentioned alarm caused by the detection of persons, a new alarm condition is customized: close contact alarm, that is, if two persons among the detected persons meet the close contact alarm condition (for example, the distance between the two persons is less than or equal to the threshold), an alarm message is generated, and the two persons meeting the close contact alarm condition are taken as alarm targets, and the alarm message and the trajectories of the two alarm targets are output as target analysis results.

[0069] In fact, the human close contact alarm here can be realized in the indoor target detection alarm unit, or the indoor target detection alarm unit can only realize the human detection alarm, and the subsequent post-processing unit can further realize the further judgment of the close contact alarm based on the information of the human detection alarm.

[0070] Step 203: The structured target detection unit processes the input surveillance video and outputs each detected target and the trajectory of the target to a post-processing module.

[0071] As mentioned above, the structured target detection unit is usually used to detect, track and obtain structured information of targets in surveillance videos. It can distinguish different targets in surveillance videos, track each target one by one to obtain the moving trajectory of the target, and can further obtain the structured information of the target, such as height, posture and other information. The target detected by the structured target detection unit is called a structured target. In this embodiment, the structured target detection unit is used to detect personnel and obtain the trajectory and structured information of each detected person. The specific methods of performing target detection and tracking and obtaining structured information can be implemented using various existing methods.

[0072] The processing of step 202 and step 203 may be performed in parallel.

[0073] Step 204, the post-processing module matches the trajectory of the alarm target corresponding to each alarm information output by the indoor target detection alarm unit with the trajectory of each structured target output by the structured target detection unit, and determines the trajectory of the matching structured target and the corresponding output structured target.

[0074] In this step, the different target analysis results output by the indoor target detection alarm unit and the structured target detection unit are matched. In this embodiment, the matching is performed in units of alarm information, and whether they belong to the same target is used as the matching standard.

[0075] Specifically, the matching is performed in units of alarm information, that is, a set of matching is performed for each alarm information to determine the corresponding matching result, and the matching process performed for each alarm information is the same. The following is an example of matching for alarm information A. When performing the matching process, the alarm target of alarm information A is matched with the structured target detected by the structured target detection unit based on whether they belong to the same target. The specific method for determining whether a match is made is:

[0076] For alarm information A, the trajectory of the alarm target corresponding to alarm information A is compared with the trajectories of each structured target detected by the structured target detection unit. Among the trajectories of all structured targets, the trajectory of the structured target with the best matching degree with the trajectory of the alarm target is found. It is considered that the trajectory of the structured target matches the trajectory of the alarm target, that is, the corresponding structured target matches the alarm target, and it can be considered that the corresponding structured target and the alarm target point to the same target, and the matching is completed. The matched structured target is called the output structured target.

[0077] It should be noted here that, in this embodiment, the alarm information corresponds to two alarm targets. Therefore, when performing matching processing, it is necessary to perform a matching operation on each of the two alarm targets to find the corresponding matching output structured target, and finally find two output structured targets that match the two alarm targets.

[0078] In addition, in the above matching process, when comparing the trajectory of the alarm target with the trajectory of the structured target to determine the trajectory of the structured target with the best match, in addition to the fitting similarity of the trajectory itself, it is also necessary to compare whether the position of the trajectory in the image is consistent, and whether the timestamp information corresponding to the trajectory points at the same position is consistent. Only when the consistency of the three is greater than or equal to the corresponding set threshold (that is, the fitting similarity of the trajectory is greater than or equal to the set trajectory similarity threshold, the position similarity of the trajectory in the image is greater than or equal to the set position similarity threshold, and the similarity of the timestamp information corresponding to the trajectory points at the same position is greater than or equal to the time similarity threshold), the corresponding trajectories are considered to be matched.

[0079] The fitting similarity of the trajectory itself can be determined by various existing trajectory comparison methods. In this embodiment, the specific comparison method can be based on an open source fitting comparison algorithm - a curve / shape similarity algorithm based on Frechet distance.

[0080] In more detail, let X and Y be Banach spaces, Ω be an open convex subset of X, x0∈Ω, f:Ω→Y. If there exists an n-order F-derivative operator f(n) of f on Ω, then for any h∈X, x0+h∈Ω, the following Taylor formula holds:

[0081]

[0082] and

[0083]

[0084] If f(n) is still continuous on Ω, then we have

[0085]

[0086] In mathematics, Taylor's formula is a formula that uses information about a function at a point to describe its nearby values. It is a method of approximating a function f(x) with an nth-order derivative at x=x0 using an nth-order polynomial about (x-x0).

[0087] If a function f(x) has an nth-order derivative on a closed interval [a,b] containing x0, and an n+1th-order derivative on an open interval (a,b), then for any point x on the closed interval [a,b], the following holds:

[0088]

[0089] Among them, f(x) represents the nth-order derivative of f(x), the polynomial after the equal sign is called the Taylor expansion of the function f(x) at x0, and the remaining Rn(x) is the remainder of the Taylor formula, which is a high-order infinitesimal of (x-x0)n.

[0090] Those skilled in the art will know that based on the above mathematical formula, the similarity between any small arcs on the two trajectory curves and the similarity of the cutting angles can be determined, and based on these similarities, the similarity of the two trajectory curves can be calculated, thereby determining the degree of curve fitting of the two trajectories themselves. Of course, determining the curve fitting similarity of the two trajectories can also be performed in other existing ways, and this application does not limit this.

[0091] The comparison and similarity calculation of the position of the entire trajectory in the image and the timestamp information of the trajectory points at the same position can be achieved using existing methods, which will not be described here.

[0092] Trajectories that are similar in terms of the degree of fit of the trajectory curve, the position of the trajectory in the image, and the timestamp information of the trajectory points at the same position, mean that the two targets corresponding to the two trajectories have the same moving trajectory at the same position within the same period of time, so it can be considered that the two targets are actually the same target. In other words, the output structured target and the alarm target obtained by matching the above trajectory comparison method point to the same target.

[0093] Step 205: Use the alarm information, the alarm target, and the structured detection result of the output structured target as the best target analysis result.

[0094] After determining the output structured target, the structured detection result of the output structured target gives the detailed structural information of the corresponding target, which can also be regarded as the detailed structural information of the alarm target. In order to meet the demand for obtaining detailed information of the alarm target, the alarm information, alarm target and the structured detection result of the output structured target are jointly used as the optimal target analysis result, which is used to characterize the alarm information and the alarm target and their detailed information.

[0095] Step 206: Based on the structured detection result in the best target analysis result, output structured target face modeling.

[0096] The optimal target analysis result includes the structured detection result of the output structured target, which is also considered as the structured detection result of the alarm target. Since the present embodiment needs to perform face retrieval of the alarm target, it is necessary to first perform face modeling of the alarm target, and then perform face retrieval using the face model generated by the modeling. Therefore, face modeling of the output structured target, that is, face modeling of the alarm target, is required in this step.

[0097] Specifically, since the structured information includes detailed information of the target, including detailed information of the face, the structured information can be used to model the face. The specific modeling method can adopt various existing methods, which will not be described here.

[0098] Step 207, based on the result of face modeling, search in the face database to obtain the face retrieval result of the output structured target, and save the result of face modeling in the face database.

[0099] The result of face modeling in step 206 is a face model, which is compared with the face model stored in the face library to determine the face search result. At the same time, in order to expand the scale of face models in the face library and improve the accuracy of subsequent retrieval of the same face, the face model obtained by face modeling in step 206 can also be stored in the face library. In this way, when the retrieval of the same face is triggered for the second time, the information of the current target can be detected more efficiently, thereby improving the accuracy of the next detection of the same face.

[0100] Step 208: Output each alarm message output by the indoor detection alarm unit and its corresponding alarm target and the face retrieval result of the output structured target accordingly.

[0101] The analysis requirement of this embodiment is to be able to perform indoor personnel detection alarm, determine the alarm target, and perform face retrieval of the alarm target. Therefore, when the result is finally output, the alarm information, the alarm target, and the face retrieval result of the output structured target are output accordingly. In addition, as mentioned above, the alarm target corresponding to a single alarm information in this embodiment is two targets in close contact. Therefore, when performing face modeling and face retrieval for the corresponding output structured target, two face models are also modeled to obtain two face models, and face retrieval is performed on the two face models to obtain two face retrieval results, which correspond to the two alarm targets respectively. Therefore, in this embodiment, the output corresponding to the alarm information A includes the face retrieval results of two alarm targets and two output structured targets.

[0102] Each alarm message is processed in the manner of steps 204-208.

[0103] So far, Figure 2The target analysis method process in the specific embodiment of the present application is ended.

[0104] In the above specific embodiment, the target analysis method is specifically described by taking the indoor target detection alarm unit and the structured target detection unit for target analysis. In fact, the method of this application can also be applied to other complex analysis requirements. For example, if the analysis requirement is: to perform perimeter detection personnel alarm and obtain the alarm target and its face retrieval results, then the above method can also be used accordingly. Figure 2 The method similar to that shown in the figure can be specifically used to perform personnel detection alarm using the perimeter detection alarm unit to determine the trajectory of the alarm target. For the trajectory of the alarm target corresponding to each alarm message, the trajectory of the matching structured target is selected from the trajectory of the structured target output by the structured target detection unit, and the structured target corresponding to the trajectory of the selected structured target is used as the output structured target corresponding to the alarm message. The structured detection result of the output structured target is used to perform face modeling and face retrieval, and the face retrieval result is output in correspondence with the alarm message and the alarm target. Among them, the trajectory comparison and matching processing is the same as Figure 2 The methods shown can be the same. Here is an application example of trajectory comparison and matching:

[0105] like Figure 3 and Figure 4 As shown, Figure 3 The detection result of the perimeter target detection alarm unit is obtained by analyzing a surveillance video through a tracking rule to obtain a trajectory segment 1 of the alarm target; Figure 4 are multiple targets detected by the structured target detection unit and their corresponding target trajectory segments 1 to 5; the optimized Fréchet algorithm is used to Figure 3 The trajectory segments in Figure 4 Compare each trajectory segment in one by one, so that Figure 4 Get the corresponding Figure 3 The trajectory segment that matches the trajectory segment in the image is obtained, and the structured detection result data of the target corresponding to the trajectory segment is sent to face modeling for subsequent face modeling and face retrieval. In more detail, Figure 4 There are 5 trajectory segments in the detection results shown. Figure 3 There is one trajectory segment in the detection results shown. Figure 4 The five trajectory segments in the detection results shown correspond to the face and body data of their respective targets. Figure 3 The trajectory segments in Figure 4 Each trajectory segment in is compared by using the optimized Fréchet algorithm to determine Figure 4 The trajectory segment 5 and Figure 3 The similarity of the middle trajectory segment is the highest, and then Figure 4 The face data and body data corresponding to trajectory segment 5 in the detection result are sent to the modeling and comparison algorithm to obtain the attributes of the current face, and face modeling and face retrieval are performed based on the current material data.

[0106] In addition, in the above Figure 2 In the process shown and the application examples above, the processing is based on the alarm information output by the target detection alarm unit. In fact, if the target detection alarm unit does not output the alarm information, the structured target detection unit can also be used to perform target retrieval to expand the target model library. Specifically, a structured target can be selected from all structured targets output by the structured target detection unit; based on the structured detection result of the selected structured target, target modeling is performed, and the result of target modeling is saved in the target model library.

[0107] The foregoing Figure 2 The specific embodiments and application examples shown provide two methods for implementing complex analysis requirements and their specific target analysis. In addition, there may be further other analysis requirements based on multiple complex analysis requirements. Specifically, there may be multiple target detection alarm units, and the alarm events of different target detection alarm units are different; the alarm target trajectory information output by each target detection alarm unit can be matched with the target trajectory information output by the structured target detection unit to determine the first target that successfully matches; for each target detection alarm unit, target modeling and target retrieval are performed on the first target to obtain respective target retrieval results; the target retrieval results of the first target corresponding to each target retrieval alarm unit are matched and compared, and the best matching target retrieval result and its corresponding alarm information are output.

[0108] For example, to satisfy Figure 2 The analysis result of the analysis requirement is an alarm message and a face search result. The analysis result that meets the analysis requirement of the application example is also an alarm message and a face search result. It may be necessary to compare the two face search results to achieve further result analysis. For this requirement, the target detection alarm unit can be set to include an indoor target detection alarm unit and a perimeter detection alarm unit. For the indoor target detection alarm unit, it can be set according to the above Figure 2The method shown is used to match and determine the output structured target (hereinafter referred to as the first output structured target); for the perimeter detection alarm unit, matching can be performed according to the method shown in the above application example to determine the corresponding output structured target (hereinafter referred to as the second output structured target); the face retrieval results of each first output structured target corresponding to each alarm information output by the indoor target detection alarm unit are matched and compared with the face retrieval results of the second output structured target corresponding to each alarm information output by the perimeter detection alarm unit, and the best matching face retrieval result and its corresponding alarm information are output or further processed.

[0109] The above is the specific implementation of the target analysis method in this application. This target analysis method can effectively solve the problem that a single target analysis process cannot meet complex needs. It can match a variety of different types of target analysis results through certain logical processing to obtain the best target analysis result, and directly output it or perform subsequent processing according to the analysis requirements to obtain the final analysis result.

[0110] The present application also provides a target analysis device, which can be used to implement the target analysis method in the present application. Figure 5 This is a schematic diagram of the basic structure of the target analysis device in this application, such as Figure 5 As shown, the device includes: at least two target analysis modules, a post-processing module and an output module.

[0111] Each target analysis module is used to process the input image data to be processed, obtain target analysis results, and output them to the post-processing module; each target analysis result includes at least one target and its trajectory information, and at least one target analysis result includes structured information of the target;

[0112] A post-processing module, used to match the trajectory information of the targets in the target analysis results obtained by the different types of target analysis modules, and determine the first target corresponding to the successfully matched trajectory information in the corresponding target analysis results;

[0113] An output module, used for outputting the result of target analysis based on the structured information of the first target;

[0114] Each target analysis module performs different types of target analysis processing.

[0115] Optionally, the image data to be processed may be a group of image frames;

[0116] The target analysis module can be respectively a target detection alarm unit and a structured target detection unit;

[0117] The target detection alarm unit can be used to process the image data, detect the alarm event and the alarm target associated with the alarm event, and output the alarm information indicating the occurrence of the alarm event and the alarm target and its trajectory information;

[0118] A structured target detection unit is used to process image data, detect structured information of the target, and output the structured target and its structured information and trajectory information;

[0119] In the post-processing module, the targets in the target analysis results obtained by the different types of target analysis modules are processed for trajectory information matching, which may specifically include:

[0120] For the alarm target corresponding to each alarm information output by the target detection alarm unit, the trajectory of the alarm target is matched with the trajectory of each structured target output by the structured target detection unit, wherein the structured target is the target in the target analysis result output by the structured target detection unit.

[0121] Optionally, the method of determining whether the match is successful may specifically include:

[0122] Based on the time information in the image data, among the trajectory information of all structured targets, the trajectory information of the structured targets that matches the trajectory of the alarm target in the same time period is determined as the trajectory information that has been successfully matched.

[0123] Optionally, the processing of outputting the result based on the structured information of the first target may specifically include:

[0124] Perform secondary analysis based on the structured information of the first target, and output results based on the secondary analysis results;

[0125] Alternatively, the structured information of the first target is directly output as a result.

[0126] Optionally, the apparatus may further include a target modeling module and a target retrieval module;

[0127] The target modeling module can be used to perform target modeling of the output structured target based on the structured detection result of the output structured target;

[0128] The target retrieval module can be used to search in the target model library based on the target modeling results, obtain the target retrieval results of the output structured target, and save the target modeling results in the target model library;

[0129] In the output module, the secondary analysis process based on the structured information of the first target may specifically include:

[0130] Each alarm information output by the target detection alarm unit and the associated alarm target and the face retrieval result of the first target are correspondingly output.

[0131] Optionally, there may be multiple target detection alarm units, and the alarm events of different target detection alarm units are different;

[0132] In the post-processing unit, the alarm target trajectory information output by each target detection alarm unit is matched with the target trajectory information output by the structured target, the first target that successfully matches is determined, and target modeling and target retrieval are performed on each first target to obtain a target retrieval result;

[0133] The device may further include a secondary comparison unit for matching and comparing target retrieval results of the first target corresponding to different target detection alarm units, and outputting the best matching target retrieval result and its corresponding alarm information through the output unit.

[0134] Optionally, the target detection alarm unit may be an indoor target detection alarm unit, the determined alarm information may be a close contact alarm, the alarm targets corresponding to the alarm information may be two targets detected that meet the close contact condition, and the target modeling may be face modeling;

[0135] In the post-processing unit, when the trajectory of the alarm target is matched with the trajectory of each structured target respectively, the matching can be performed for each of the two targets; the alarm information corresponds to two output structured targets;

[0136] In the face modeling module, face modeling is performed for each of the two output structured targets;

[0137] In the face retrieval module, a search is performed for each of the two output structured targets to obtain face retrieval results corresponding to the two output structured targets;

[0138] In the output module, the face retrieval results of each alarm information and its corresponding two output structured targets are output accordingly.

[0139] Optionally, in the post-processing module, the process of matching the trajectory of the alarm target with the trajectory of each structured target output by the structured target detection unit may specifically include:

[0140] The trajectory of the alarm target is compared with the trajectories of each structured target respectively. Based on the curve fitting effect of the trajectory, the degree of overlap of the trajectory positions in the entire frame image, and the degree of consistency of the timestamp information corresponding to the trajectory points at the same position, the trajectory of the structured target that matches the trajectory of the alarm target is selected from the trajectories of each structured target.

[0141] Optionally, when the target detection alarm unit does not output alarm information, the post-processing unit may be further configured to select a structured target from all structured targets output by the structured target detection unit;

[0142] The face modeling module can be used to perform target modeling based on the structured detection results of the structured targets selected by the post-processing unit, and save the target modeling results in a target model library.

[0143] The present application also provides a computer-readable storage medium that stores instructions, which, when executed by a processor, can execute the steps in the target analysis method as described above. In practical applications, the computer-readable medium can be included in each device / apparatus / system in the above-mentioned embodiments, or it can exist independently without being assembled into the device / apparatus / system. Instructions are stored in a computer-readable storage medium, and the instructions stored therein can execute the steps in the target analysis method as described above when executed by a processor.

[0144] According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, but it is not used to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0145] Figure 6 This application also provides an electronic device. Figure 6 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0146] The electronic device may include a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 602 is executed, the target analysis method may be implemented.

[0147] Specifically, in practical applications, the electronic device may further include components such as a power supply 603, an input / output unit 604, etc. Those skilled in the art may understand that Figure 6The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0148] The processor 601 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 602 and calling data stored in the memory 602, thereby monitoring the electronic device as a whole.

[0149] The memory 602 can be used to store software programs and modules, that is, the above-mentioned computer-readable storage medium. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0150] The electronic device also includes a power supply 603 for supplying power to various components, which can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 603 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0151] The electronic device may also include an input and output unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, and optical signal inputs related to user settings and function control. The input unit output 604 may also be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof.

[0152] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A target analysis method, characterized in that: include: Inputting the image data to be processed into at least two different types of target analysis modules; Each of the target analysis modules processes the image data respectively to obtain respective target analysis results; wherein each of the target analysis results includes at least one target and its trajectory information, and at least one target analysis result includes structural information of the target; Matching the trajectory information of the targets in the target analysis results obtained by the different types of target analysis modules, and determining the first target corresponding to the successfully matched trajectory information in the corresponding target analysis results; Based on the structured information of the first target, output the result; The target analysis module includes a target detection alarm unit and a structured target detection unit; the target detection alarm unit is used to detect an alarm event and an alarm target associated with the alarm event, and output alarm information indicating the occurrence of the alarm event and the alarm target and its trajectory information; the structured target detection unit is used to detect the structured information of the target, and output the structured target and its structured information and trajectory information; The matching of trajectory information of targets in target analysis results obtained by different types of target analysis modules includes: for each alarm target output by the target detection alarm unit, matching the trajectory information of the alarm target with the trajectory information of each structured target output by the structured target detection unit.

2. The method according to claim 1, characterized in that Methods for determining whether a match is successful include: Based on the time information in the image data, among the trajectory information of all structured objects, the trajectory information of the structured objects that matches the trajectory of the alarm object in the same time period is determined as the trajectory information that has been successfully matched.

3. The method according to claim 2, characterized in that The outputting of results based on the structured information of the first target includes: Performing secondary analysis based on the structured information of the first target, and outputting results based on the secondary analysis results; Alternatively, the structured information of the first target is directly output as a result.

4. The method according to claim 3, characterized in that The performing secondary analysis based on the structured information of the first target includes: Based on the structured information of the first target output by the structured target detection unit, performing target modeling of the output first target; Based on the result of the target modeling, searching in the target model library to obtain the target search result of the first target, and saving the result of the target modeling in the target model library; Each alarm information output by the target detection alarm unit and the associated alarm target and the target retrieval result of the first target are correspondingly output.

5. The method according to claim 4, characterized in that There are multiple target detection alarm units, and the alarm events of different target detection alarm units are different; Matching the alarm target trajectory information output by each of the target detection alarm units with the target trajectory information output by the structured target detection unit, determining a first target that successfully matches, and performing target modeling and target retrieval on each of the first targets to obtain the target retrieval result; The method further comprises: The target retrieval results of the first target corresponding to different target detection alarm units are matched and compared, and the best matching target retrieval result and its corresponding alarm information are output.

6. The method according to claim 4, characterized in that The target detection alarm unit is an indoor target detection alarm unit, the determined alarm information is a close contact alarm, the alarm targets associated with the alarm information are two detected targets that meet the close contact condition, and the target modeling is human face modeling; When the trajectory information of the alarm target is matched with the trajectory information of each of the structured targets respectively, the matching is performed for each of the two targets; the alarm information is associated with the two alarm targets; For each of the two output structured targets, perform face modeling and the retrieval to obtain face retrieval results for each of the two output structured targets; The face retrieval results of each alarm information and its corresponding two output structured targets are output accordingly.

7. The method according to claim 2, characterized in that The step of determining the trajectory information of the structured target having the highest matching degree with the trajectory of the alarm target in the same time period as the successfully matched trajectory information includes: The trajectory information of the alarm target is compared with the trajectory information of each structured target respectively. Based on the curve fitting effect of the trajectory, the degree of overlap of the trajectory positions in the entire frame image, and the degree of consistency of the timestamp information corresponding to the trajectory points at the same position, the trajectory of the structured target that matches the trajectory of the alarm target is selected from the trajectories of the various structured targets.

8. The method according to claim 4, characterized in that When the target detection alarm unit does not output alarm information, the method further includes: Selecting a structured object from all structured objects output by the structured object detection unit; Based on the structured information of the selected structured target, target modeling is performed, and the result of the target modeling is stored in the target model library.

9. An electronic device, characterized in that: The electronic device includes at least a computer-readable storage medium and also includes a processor; The processor is used to read executable instructions from the computer-readable storage medium and execute the instructions to implement the target analysis method described in any one of claims 1 to 8 above.

Images