A method, apparatus and electronic device for analyzing a critical event

Abstract

The present disclosure provides a key event analysis method, device and electronic equipment, relates to the technical field of video event analysis method, and solves the problem that in the prior art, the corresponding key event can be marked only after the information to be analyzed is viewed one by one by the viewer when the key event is analyzed, resulting in low key event analysis efficiency. The method comprises the following steps: acquiring a first time sequence feature pool of a first video interval of a to-be-processed video and target data flow; acquiring a second time sequence feature pool in the target data flow based on the first time sequence feature pool; determining a second video interval based on the second time sequence feature pool; and determining a target video interval corresponding to a key event in the to-be-processed video based on the first video interval and the second video interval.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of video event analysis method, in particular to a key event analysis method, device and electronic equipment. BACKGROUND

[0002] At present, in the analysis scene of key events, it is necessary to rely on the viewing personnel to check the information to be analyzed, and then find out the corresponding key events. For example, when analyzing the key events of video monitoring, the viewing personnel needs to watch the video monitoring and mark the corresponding key events.

[0003] It can be seen that in the prior art, when analyzing key events, the viewing personnel needs to view the information to be analyzed one by one, and then mark the corresponding key events, which leads to low efficiency of key event analysis. SUMMARY

[0004] In order to solve the above technical problems, the present disclosure provides a key event analysis method, device and electronic equipment.

[0005] In order to achieve the above purpose, the present disclosure provides the technical scheme as follows:

[0006] In a first aspect, the present disclosure provides a key event analysis method, comprising: acquiring a first time sequence feature pool of a to-be-processed video in a first video interval, and a target data stream; wherein the target data stream is collected in the same physical space as the to-be-processed video; based on the first time sequence feature pool, acquiring a second time sequence feature pool in the target data stream; based on the second time sequence feature pool, determining a second video interval; based on the first video interval and the second video interval, determining a target video interval corresponding to a key event in the to-be-processed video.

[0007] In some implementable examples, based on the first time sequence feature pool, the second time sequence feature pool in the target data stream is acquired, comprising: digitally encoding the target data stream to obtain first encoded data; in the first encoded data, acquiring second encoded data aligned with the first time sequence feature pool; in the first encoded data, acquiring third encoded data, the third encoded data having effective features in the second encoded data;

[0008] The second time sequence feature corresponding to the third encoded data is acquired.

[0009] In some implementable examples, in the first encoded data, the third encoded data is acquired, comprising: using a preset feature analysis model to learn the second encoded data; the feature analysis model is used to make classification decisions on the distribution characteristics of the encoded data in different frequency domains; using the learned feature analysis model to process the first encoded data to obtain the third encoded data.

[0010] In some implementable examples, the feature analysis model is a gradient boosting decision tree model.

[0011] In some implementable examples, based on the first time sequence feature pool, a second time sequence feature pool in the target data stream is obtained, including: digitally encoding the target data stream to obtain first encoded data; in the first encoded data, second encoded data aligned with the first time sequence feature pool is obtained; and a second time sequence feature pool corresponding to the second encoded data is obtained.

[0012] In some implementable examples, based on the second time sequence feature pool, a second video interval of the key event is determined, including: based on the second time sequence feature pool and the to-be-processed video, the second video interval of the key event is determined.

[0013] In some implementable examples, the second video interval includes a first candidate interval and a second candidate interval; based on the second time sequence feature pool, the second video interval is determined, including: based on the second time sequence feature pool and the to-be-processed video, the first candidate interval is determined; and based on the second time sequence feature pool and the reference video, the second candidate interval is determined.

[0014] In some implementable examples, based on the first video interval and the second video interval, a target video interval corresponding to the key event in the to-be-processed video is determined, including: feature similarity analysis is performed on video data of the first video interval and the second video interval to determine an analysis result; and based on the analysis result, the target video interval corresponding to the key event in the to-be-processed video is determined.

[0015] In some implementable examples, the feature similarity analysis is performed on the video data of the first video interval and the second video interval to determine the analysis result, including: based on a pre-configured video encoding algorithm, the video data of the first video interval and the second video interval is encoded; similarity data between video encoding data of the second video interval and video encoding data of the first video interval is obtained to obtain the analysis result.

[0016] In some implementable examples, the pre-configured video encoding algorithm is an optical flow network algorithm.

[0017] In some implementable examples, based on the analysis result, the target video interval corresponding to the key event in the to-be-processed video is determined, including: based on the similarity data, a third video interval is determined, the third video interval being a second video interval in which a total number of the similarity data is greater than or equal to a preset threshold; and based on the first video interval and the third video interval, the target video interval corresponding to the key event in the to-be-processed video is determined.

[0018] In some embodiments, the target data stream includes at least one of an audio data stream, a video data stream, and a sensing data stream, wherein the sensing data stream is collected by a sensing device including at least one of an acoustic sensor, an ultrasonic sensor, an electromagnetic wave sensor, an infrared sensor, an infrared camera, an optical sensor, a mechanical sensor, and a temperature sensor.

[0019] In a second aspect, the disclosure provides a key event analysis device, including: an acquisition unit configured to acquire a first time sequence feature pool of a first video interval of a to-be-processed video and a target data stream, wherein the target data stream is collected in the same physical space as the to-be-processed video; a processing unit configured to determine a second time sequence feature pool in the target data stream acquired by the acquisition unit based on the first time sequence feature pool acquired by the acquisition unit; the processing unit is further configured to determine a second video interval based on the second time sequence feature pool; and the processing unit is further configured to determine a target video interval corresponding to a key event in the to-be-processed video based on the first video interval and the second video interval.

[0020] In some embodiments, the processing unit is specifically configured to digitally encode the target data stream acquired by the acquisition unit to obtain first encoded data; the processing unit is specifically configured to acquire second encoded data aligned with the first time sequence feature pool in the first encoded data; the processing unit is specifically configured to acquire third encoded data in the first encoded data, the third encoded data having effective features in the second encoded data; and the processing unit is specifically configured to acquire the second time sequence feature corresponding to the third encoded data.

[0021] In some embodiments, the processing unit is specifically configured to learn the second encoded data using a preset feature analysis model; the feature analysis model is used for classification decision on the distribution features of the encoded data in different frequency domains; and the processing unit is specifically configured to process the first encoded data using the learned feature analysis model to obtain the third encoded data.

[0022] In some embodiments, the feature analysis model is a gradient boosting decision tree model.

[0023] In some embodiments, the processing unit is specifically configured to digitally encode the target data stream acquired by the acquisition unit to obtain first encoded data; the processing unit is specifically configured to acquire second encoded data aligned with the first time sequence feature pool in the first encoded data; and the processing unit is specifically configured to acquire the second time sequence feature pool corresponding to the second encoded data.

[0024] In some embodiments, the processing unit is specifically configured to determine the second video interval of the key event based on the second time sequence feature pool and the to-be-processed video acquired by the acquisition unit.

[0025] In some embodiments, the second video interval includes a first candidate interval and a second candidate interval; the processing unit is specifically configured to determine the first candidate interval based on the second time sequence feature pool and the to-be-processed video obtained by the obtaining unit; the processing unit is specifically configured to determine the second candidate interval based on the second time sequence feature pool and a reference video; and the reference video is historical video data collected by a video collection device of the to-be-processed video.

[0026] In some embodiments, the processing unit is specifically configured to perform feature similarity analysis on the video data of the first video interval and the second video interval to determine an analysis result; and the processing unit is specifically configured to determine the target video interval corresponding to the key event in the to-be-processed video based on the analysis result.

[0027] In some embodiments, the processing unit is specifically configured to encode the video data of the first video interval and the second video interval based on a preconfigured video encoding algorithm; and the processing unit is specifically configured to obtain the analysis result according to similarity data between the video encoding data of the second video interval and the video encoding data of the first video interval obtained by the obtaining unit.

[0028] In some embodiments, the preconfigured video encoding algorithm is an optical flow network algorithm.

[0029] In some embodiments, the processing unit is specifically configured to determine a third video interval based on the similarity data, the third video interval being a second video interval whose total number of similarity data is greater than or equal to a preset threshold; and the processing unit is specifically configured to determine the target video interval corresponding to the key event in the to-be-processed video based on the first video interval and the third video interval.

[0030] In some embodiments, the target data stream includes at least one of an audio data stream, a video data stream, and a sensing data stream; and the sensing data stream is obtained by a sensing device, and the sensing device includes at least one of an acoustic sensor, an ultrasonic sensor, an electromagnetic wave sensor, an infrared sensor, an infrared camera, an optical sensor, a mechanical sensor, and a temperature sensor.

[0031] In a third aspect, the present disclosure provides an electronic device, including a communication interface, a processor, a memory, and a bus; the memory is configured to store computer execution instructions; the processor is connected to the memory through the bus. When the electronic device is running, the processor executes the computer execution instructions stored in the memory, so that the electronic device executes the key event analysis method provided in the first aspect.

[0032] In a fourth aspect, the present disclosure provides a computer-readable storage medium, including instructions. When the instructions are run on a computer, the computer executes the key event analysis method provided in the first aspect.

[0033] In a fifth aspect, the present disclosure provides a computer program product, which, when running on a computer, causes the computer to execute the key event analysis method of the design manner of the first aspect.

[0034] It should be noted that the computer instructions described above can be stored in whole or in part on the first computer readable storage medium. The first computer readable storage medium can be packaged together with the processor of the key event analysis apparatus, or can be packaged separately from the processor of the key event analysis apparatus, and the present disclosure does not limit this.

[0035] The second aspect, the third aspect, the fourth aspect and the fifth aspect of the present disclosure can refer to the detailed description of the first aspect; and the beneficial effects of the second aspect, the third aspect, the fourth aspect and the fifth aspect can refer to the beneficial effect analysis of the first aspect, which will not be described here.

[0036] In the present disclosure, the name of the key event analysis apparatus described above does not constitute a limitation on the device or functional module itself, and in actual implementation, these devices or functional modules can appear with other names. As long as the functions of each device or functional module are similar to the present disclosure, it belongs to the scope of the claims of the present disclosure and its equivalent technology.

[0037] These aspects or other aspects of the present disclosure will be more apparent in the following description.

[0038] The technical solutions provided by the present disclosure have the following advantages compared with the prior art:

[0039] In analyzing the key event of the to-be-processed video, the key event of the to-be-processed video can be analyzed by combining the time sequence features of the target data stream collected in the same physical space as the to-be-processed video. For example, by obtaining a first time sequence feature pool in a first video interval of the to-be-processed video, a second time sequence feature pool in the target data stream can be further obtained based on the first time sequence feature pool. Then, a second video interval is determined by the second time sequence feature pool and the to-be-processed video. In this way, the second video interval related to the second time sequence feature pool in the to-be-processed video can be analyzed by combining the target data stream collected in the same physical space. Then, the target video interval corresponding to the key event in the to-be-processed video is determined based on the first video interval and the second video interval. As can be seen, by using the key event analysis method provided by the present disclosure, it is not necessary to continuously watch the to-be-analyzed video by artificial to capture the key event of the to-be-analyzed video, but the device executing the key event analysis method provided by the present disclosure is used to analyze the to-be-processed video, thereby improving the analysis efficiency of the key event. The problem of low analysis efficiency of the key event in the prior art is solved, which requires the viewer to view the information to be analyzed one by one before marking the corresponding key event. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate an embodiment consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0042] Figure 1 A scene diagram of the key event analysis method provided by the embodiment of the present disclosure is shown in the following figure:

[0043] Figure 2 A flowchart of the key event analysis method provided by the embodiment of the present disclosure is shown in the following figure:

[0044] Figure 3 A flowchart of the key event analysis method provided by the embodiment of the present disclosure is shown in the following figure:

[0045] Figure 4 A flowchart of the key event analysis method provided by the embodiment of the present disclosure is shown in the following figure:

[0046] Figure 5 A flowchart of the key event analysis method provided by the embodiment of the present disclosure is shown in the following figure:

[0047] Figure 6 Fig. 5 is a flowchart of a key event analysis method according to an embodiment of the present disclosure;

[0048] Figure 7 Fig. 6 is a logic diagram of a key event analysis method according to an embodiment of the present disclosure;

[0049] Figure 8 Fig. 7 is a structural diagram of a server according to an embodiment of the present disclosure;

[0050] Figure 9 Fig. 8 is a structural diagram of a server according to an embodiment of the present disclosure;

[0051] Figure 10 Fig. 9 is a structural diagram of a computer program product of a key event analysis method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0053] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other different manners from those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present disclosure, and not all the embodiments.

[0054] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a..." does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0055] The timestamp in the embodiments of the present disclosure refers to data generated using digital signature technology, and the signed objects include original file information, signature parameters, signature time and the like.

[0056] Figure 1 Fig. 1 is a scenario diagram of a key event analysis method according to an embodiment of the present disclosure. As shown in Fig. 1, the key event analysis method according to an embodiment of the present disclosure can be applied to a server.Figure 1 As shown in (a), the system includes a server 1, an image acquisition device 2, and an audio acquisition device 3. The image acquisition device 2 and the audio acquisition device 3 are located in the same physical space. The server 1 receives data acquired by both the image acquisition device 2 and the audio acquisition device 3. When performing critical event analysis on the physical space where the image acquisition device 2 is located, the server 1 can combine the data acquired by the audio acquisition device 3 with the data acquired by the image acquisition device 2 to analyze the critical events in the video to be processed.

[0057] Or, such as Figure 1 As shown in (b), the system includes a server 1, an image acquisition device 2, and a sensing device 4. The image acquisition device 2 and the sensing device 4 are located in the same physical space. The server 1 receives data acquired by both the image acquisition device 2 and the sensing device 4. When performing critical event analysis on the physical space where the image acquisition device 2 is located, the server 1 can combine the data acquired by the sensing device 4 with the data acquired by the image acquisition device 2 to analyze the critical events in the video to be processed.

[0058] Or, such as Figure 1 As shown in (c), the system includes a server 1, an image acquisition device 2, a sound acquisition device 3, and a sensing device 4. The image acquisition device 2, sound acquisition device 3, and sensing device 4 are located in the same physical space. The server 1 receives data collected by the image acquisition device 2, sound acquisition device 3, and sensing device 4. When performing critical event analysis on the physical space where the image acquisition device 2 is located, the server 1 can combine the data collected by the sound acquisition device 3 and sensing device 4 to analyze the data collected by the image acquisition device 2, thereby identifying the critical events in the video to be processed.

[0059] Specifically, image acquisition device 2 may include, but is not limited to, image acquisition devices or video acquisition devices, such as surveillance cameras. Sound acquisition device 3 may be an audio acquisition device, such as a microphone or pickup. Sensing device 4 may include, but is not limited to, one or more of the following: acoustic sensors, ultrasonic sensors, electromagnetic wave sensors, infrared sensors, infrared cameras, optical sensors, mechanical sensors, and temperature sensors.

[0060] Figure 2 This is a flowchart illustrating a critical event analysis method according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following S11-S14.

[0061] S11. Obtain the first temporal feature pool of the video to be processed in the first video interval, and the target data stream. The target data stream and the video to be processed are acquired in the same physical space.

[0062] In some examples, the first video interval is generated by labeling a key event in the to-be-processed video. One first video interval corresponds to a first time sequence feature pool, and the timestamp combination of the first video interval is labeled. For example, the to-be-processed video is input into a pre-configured video recognition model, at least one first video interval included in the to-be-processed video is determined, and a first time sequence feature pool corresponding to each first video interval is determined. The video recognition model is trained by a sample video and a labeled result (first video interval and first time sequence feature pool corresponding to the first video interval) of the sample video.

[0063] Specifically, the timestamp combination needs to be converted into a real-world physical timestamp combination to facilitate the alignment of information at the same time in the subsequent steps.

[0064] S12, determining a second time sequence feature pool in the target data stream based on the first time sequence feature pool.

[0065] In some examples, the collection device of the target data is different from the collection device of the to-be-processed video. For example Figure 1 As shown, the collection device of the target data stream can be one or more of the sound collection device 3 or the sensing device 4 (for the convenience of description, the collection device of the target data stream can be the sound collection device 3 in the following description), and the collection device of the to-be-processed video can be the image collection device 2.

[0066] S13, determining a second video interval based on the second time sequence feature pool.

[0067] In some examples, in combination with the example given in S11 above, the first video interval is generated by labeling a key event in the to-be-processed video, and one first video interval corresponds to one first time sequence feature pool, so the second time sequence feature pool in the target data stream can be determined according to the first time sequence feature pool.

[0068] For example, in combination with Figure 1In the scenario shown in (a), the key event is an alarm event, and the target data stream is collected by the sound collection device 3. By analyzing the video collected by the image collection device 2, it is determined that an alarm event occurred between 18:00 and 19:00 on April 26, 2022, and the first time sequence feature pool is the time sequence feature in 18:00-19:00. After encoding the target data stream, when analyzing the target data stream, it can be determined that the effective feature of the target data stream between 18:00 and 19:00. When analyzing the target data stream of other time periods, it is found that the target data stream between 14:00 and 15:00 on April 26, 2022 also has this effective feature. Therefore, the second video interval can be determined as 14:00-15:00 on April 26, 2022. In this way, the video interval in the to-be-processed video that aligns with the second time sequence feature pool can be taken as the second video interval, so that the alarm event in the to-be-processed video can be analyzed.

[0069] S14, based on the first video interval and the second video interval, determining a target video interval corresponding to the key event in the to-be-processed video.

[0070] Specifically, the key event is any self-defined event occurring in the first video interval. For example, it can include but is not limited to at least one of the following: an alarm event, a reminder event, an abnormal light event, etc., which is not exhaustive.

[0071] It can be seen that by using the key event analysis method provided by the present disclosure, it is not necessary to rely on artificial continuous watching of the to-be-analyzed video to capture the key event of the to-be-analyzed video. Instead, the device executing the key event analysis method provided by the present disclosure analyzes the to-be-processed video, which improves the analysis efficiency and accuracy of the key event. The problem of low analysis efficiency of the key event in the prior art is solved, which requires the viewer to view the information to be analyzed one by one before marking the corresponding key event.

[0072] In some implementable examples, in the scenario shown in (a), the key event is an alarm event, and the target data stream is collected by the sound collection device 3. By analyzing the video collected by the image collection device 2, it is determined that an alarm event occurred between 18:00 and 19:00 on April 26, 2022, and the first time sequence feature pool is the time sequence feature in 18:00-19:00. After encoding the target data stream, when analyzing the target data stream, it can be determined that the effective feature of the target data stream between 18:00 and 19:00. When analyzing the target data stream of other time periods, it is found that the target data stream between 14:00 and 15:00 on April 26, 2022 also has this effective feature. Therefore, the second video interval can be determined as 14:00-15:00 on April 26, 2022. In this way, the video interval in the to-be-processed video that aligns with the second time sequence feature pool can be taken as the second video interval, so that the alarm event in the to-be-processed video can be analyzed. Figure 2 Figure 3 The above S12 can be implemented by the following S120-S123.

[0073] S120, digitally encoding the target data stream to obtain first encoded data.

[0074] ​In some examples, in order to unify the data format of the template data stream, the target data stream needs to be digitally encoded. For example, when the target data stream is audio data, the audio data needs to be converted into audio data of a first specified format. The first specified format can be any specified format, which is not particularly limited in the present application. For example, the first specified format can include at least one of MP3, WMA (Windows Media Audio), and the like, but is not limited to these.

[0075] For example, when the target data stream is audio data, the digital encoding includes any one of a spectrogram and a mel spectrogram.

[0076] S121, in the first encoded data, the second encoded data aligned with the first time sequence feature pool is obtained.

[0077] In some examples, the first time sequence feature pool includes at least one timestamp, and the first encoded data includes at least one timestamp. Therefore, by matching the same timestamp in the first encoded data as the first time sequence feature pool, the second encoded data aligned with the first time sequence feature pool can be obtained in the first encoded data. In this way, the time sequence alignment of the target data stream and the first video interval can be achieved.

[0078] S122, in the first encoded data, the third encoded data is obtained. The third encoded data has the effective feature in the second encoded data.

[0079] In some examples, since the first encoded data is obtained in the same physical space as the to-be-processed video, by analyzing the first encoded data and combining the analysis result, the key event of the to-be-processed video can be analyzed. For example, in the first encoded data except the second encoded data, the third encoded data having the same effective feature as the second encoded data is obtained, or in the first encoded data, the third encoded data having the effective feature in the second encoded data is obtained.

[0080] Specifically, the third encoded data can be a part of the second encoded data, or the third encoded data can be the encoded data in the first encoded data except the second encoded data and having the same effective feature as the second encoded data.

[0081] For example, in combination with the above examples, the third encoded data can be a part of the second encoded data, or the third encoded data can be the encoded data in the first encoded data except the second encoded data and having the same effective feature as the second encoded data. Figure 1In the scenario shown in (a), the key event is an alarm event, and the target data stream is collected by the sound collection device 3. By analyzing the video collected by the image collection device 2, it is determined that an alarm event occurred between 18:00 and 19:00 on April 26, 2022, and the first time sequence feature pool is the time sequence feature in the time interval between 18:00 and 19:00 on April 26, 2022. After encoding the target data stream, when analyzing the target data stream, it can be determined that the effective feature of the target data stream between 18:00 and 19:00 on April 26, 2022. When analyzing the target data stream of other time intervals, it is found that the target data stream between 14:00 and 15:00 on April 26, 2022 is very close to the target data stream between 18:00 and 19:00 on April 26, 2022, so it can be determined that the target data stream between 14:00 and 15:00 on April 26, 2022 also has the effective feature. Therefore, the target data stream between 14:00 and 15:00 on April 26, 2022 can be regarded as the third encoded data. In this way, by combining the third encoded data, all data related to the alarm event in the video to be processed can be analyzed.

[0082] S123, obtaining a second time sequence feature corresponding to the third encoded data.

[0083] Specifically, the second time sequence feature includes each timestamp corresponding to the third encoded data.

[0084] It can be seen that, in order to better analyze the key event of the video to be processed, the key event analysis method provided by the embodiment of the disclosure can analyze the target data stream collected in the same physical space as the video to be processed, so as to obtain the third encoded data with the effective feature in the second encoded data. Further, according to the third encoded data, the second time sequence feature is determined. In this way, the second video interval can be determined according to the second time sequence feature pool. Further, based on the first video interval and the second video interval, the target video interval corresponding to the key event in the video to be processed is determined, so that the key event of the video to be processed can be analyzed.

[0085] In some implementable examples, in the embodiment shown in (a) or any other embodiment described above, Figure 3 Further, the above S122 can be implemented by the following S1220 and S1221 based on the embodiment shown in (a) or any other embodiment described above.

[0086] S1220, learning the second encoded data by using a preset feature analysis model. The feature analysis model is used for making classification decisions on the distribution characteristics of the encoded data in different frequency domains.

[0087] In some examples, the preset feature analysis model needs to continuously learn the second encoding data, so as to find third encoding data in the first encoding data that has the same effective feature as the second encoding data. In other words, in this embodiment, the specific content of the effective feature does not need to be specifically distinguished or recognized, but the third encoding data with similar or the same feature as the second encoding data is obtained through feature analysis of the target data stream encoding data.

[0088] For example, the training process of the preset feature analysis model is as follows: first, obtain the training encoding data and the second encoding data. Second, input the second encoding data into the decision tree model for learning. Then, input the training encoding data into the learned decision tree model. Then, based on the target loss function, determine whether the third encoding data output by the learned decision tree model for the training encoding data has the same effective feature as the second encoding data. Thus, when the third encoding data does not have the same effective feature as the second encoding data, the network parameters of the decision tree model are repeatedly updated until the model converges, and the feature analysis model is obtained.

[0089] S1221, using the learned feature analysis model to process the first encoding data to obtain the third encoding data.

[0090] As can be seen, the preset feature analysis model is used to learn the second encoding data, so that the feature analysis model can identify other encoding data with the effective feature corresponding to the second encoding data. In this way, after the first encoding data is input into the feature analysis model, the feature analysis model can identify the third encoding data in the first encoding data with the effective feature in the second encoding data. Further, the second time sequence feature is determined according to the third encoding data. In this way, the second video interval can be determined according to the second time sequence feature pool. Further, based on the first video interval and the second video interval, the target video interval corresponding to the key event in the to-be-processed video is determined, so that the key event of the to-be-processed video can be analyzed.

[0091] It should be noted that the above examples are described by taking the example of processing the first encoding data by the learned feature analysis model to obtain the third encoding data. In some other examples, the third encoding data can also be determined by other ways. In an exemplary embodiment, the feature similarity between the first encoding data and the second encoding data can be calculated, so that the third encoding data in the first encoding data is determined according to the feature similarity. For example, the first encoding data with a feature similarity greater than or equal to a similarity threshold is taken as the third encoding data.

[0092] In some examples, the feature analysis model is a gradient boosting decision tree model.

[0093] It can be seen that the key event analysis method provided by the embodiment of the disclosure uses the second encoding data, and uses the preset feature analysis model to learn the second encoding data. In this way, after the gradient boosting decision tree model converges, when the first encoding data is analyzed, the third encoding data with the effective features in the second encoding data in the first encoding data can be determined, and the accuracy of the key event analysis is ensured.

[0094] In some implementable examples, in some implementable examples, in Figure 2 Based on the embodiments shown in the foregoing description, in addition to Figure 3 In addition to the embodiments shown in the foregoing description, the embodiment of the disclosure also provides another possible implementation manner, which can be referred to Figure 4 The S12 can be implemented by the following S124-S126.

[0095] S124, digitally encoding the target data stream to obtain first encoding data.

[0096] S125, in the first encoding data, obtaining second encoding data aligned with the first time sequence feature pool.

[0097] S126, obtaining a second time sequence feature pool corresponding to the second encoding data.

[0098] It can be seen that, in order to better analyze the key event of the to-be-processed video, the key event analysis method provided by the embodiment of the disclosure analyzes the target data stream collected in the same physical space as the to-be-processed video, so that the second encoding data aligned with the first time sequence feature pool can be obtained. Further, the second time sequence feature is determined according to the second encoding data. In this way, the second video interval can be determined according to the second time sequence feature pool. Further, based on the first video interval and the second video interval, the target video interval corresponding to the key event in the to-be-processed video is determined, so that the key event of the to-be-processed video can be analyzed.

[0099] It should be noted that the above examples are described by taking the target data stream as an example, first digitally encoding the target data stream, and then analyzing the obtained first encoding data to obtain the second time sequence feature pool. In other examples, the aligned data aligned with the first time sequence feature pool can be directly obtained in the target data stream. Then, the aligned data is digitally encoded to obtain the second encoding data. It can be seen that, compared with the above-mentioned digital encoding of the target data stream, only the aligned data is digitally encoded here, which can reduce the occupation of the computing resources and ensure the experience of the user.

[0100] In some implementable examples, in Figure 2S13 can be implemented by S130 based on the illustrated embodiment or any other preceding embodiment.

[0101] S130, based on the second time sequence feature pool and the to-be-processed video, determines a second video interval of the key event. In some examples, by obtaining a video interval in the to-be-processed video that aligns with the second time sequence feature pool, the second video interval of the key event can be determined. For example, in combination with the example given in S122, since the target data stream between 14:00-15:00 on April 26, 2022 in the target data stream has the effective features of the target data stream between 18:00-19:00 on April 26, 2022. Therefore, it can be determined that the time interval corresponding to the second time sequence feature pool is 14:00-15:00 on April 26, 2022. Further, according to the second time sequence feature pool and the to-be-processed video, the second video interval of the key event is determined to be 14:00-15:00 on April 26, 2022 in the to-be-processed video, so that the alarm event of the to-be-processed video can be analyzed.

[0102] It can be seen that, in order to better analyze the key event of the to-be-processed video, the key event analysis method provided by the embodiment of the disclosure can analyze the target data stream collected in the same physical space as the to-be-processed video, so as to obtain the second time sequence feature that can assist in analyzing the key event of the to-be-processed video. Further, by finding a video interval in the to-be-processed video that aligns with the second time sequence feature as a second video interval, the key event of the to-be-processed video can be analyzed.

[0103] In addition to the above-mentioned way of implementing S13 by S130, in some other examples, the second video interval can also be determined in combination with the reference video, and at this time, the second video interval includes the first candidate interval and the second candidate interval. For example, the reference video can be a historical video data collected by a video collection device of the to-be-processed video. Figure 5 S13 can be implemented by S131 and S132, and S14 can be implemented by S140.

[0104] S131, based on the second time sequence feature pool and the to-be-processed video, determines the first candidate interval.

[0105] S132, based on the second time sequence feature pool and the reference video, determines the second candidate interval. The reference video is historical video data collected by a video collection device of the to-be-processed video.

[0106] In some examples, to better analyze the key events in the video to be analyzed, the key event analysis method provided in this disclosure can also combine historical video data collected by the video capture device of the video to be processed to analyze the key events in the video to be analyzed, thereby improving the accuracy of key event analysis. For example, when obtaining the first temporal feature pool of the video to be processed in the first video interval, and the target data stream, a reference video collected by the video capture device of the video to be processed is also obtained. At this time, a first candidate interval can be determined based on the second temporal feature pool and the video to be processed. A second candidate interval is determined based on the second temporal feature pool and the reference video.

[0107] S140. Based on the first video interval, the first candidate interval, and the second candidate interval, determine the target video interval corresponding to the key events in the video to be processed.

[0108] As can be seen, in order to better analyze the key events in the video to be processed, the key event analysis method provided in this disclosure combines historical video data collected by the video acquisition device of the video to be processed with the target data stream collected in the same physical space of the video to be processed for analysis, thereby improving the efficiency of key event analysis. For example, in conjunction with the example given in S122 above, when analyzing the target data stream of other time periods, it was found that the target data stream between 14:00 and 15:00 on April 25, 2022 also has this effective feature. Therefore, the second time-series feature pool can be determined as the time-series feature between 14:00 and 15:00 on April 25, 2022. Further, based on the second time-series feature pool and the reference video, the second video interval of the key event is determined to be 14:00 to 15:00 on April 25, 2022 in the reference video, thereby enabling the analysis of alarm events in the video to be processed.

[0109] In some feasible instances, Figure 2 Further details can be found in the embodiments shown or any of the other embodiments described above. Figure 6 The above S14 can be implemented through the following S141 and S142.

[0110] S141. Perform feature similarity analysis on the video data of the first video interval and the second video interval, and determine the analysis results.

[0111] In some examples, there is a certain relationship between the first video interval and the second video interval. In order to better analyze the relationship between the two, the feature similarity analysis can be performed on the video data of the first video interval and the second video interval to determine the analysis result. For example, as described in the example of S122, when analyzing the target data stream of other time periods, it is found that the target data stream between 14:00-15:00 on April 25, 2022 also has the effective feature. Therefore, it can be determined that the second time sequence feature pool is the time sequence feature between 14:00-15:00 on April 25, 2022. Further, according to the second time sequence feature pool and the reference video, the second video interval of the key event is determined to be 14:00-15:00 on April 25, 2022 in the reference video. In this way, when performing feature similarity analysis on the first video interval corresponding to 18:00-19:00 on April 26, 2022 and the second video interval corresponding to 14:00-15:00 on April 25, 2022, the same features between the two can be found, and then the analysis result can be generated according to the same features. The analysis result is used to analyze the alarm event of the to-be-processed video.

[0112] S142, determining the target video interval corresponding to the key event in the to-be-processed video based on the analysis result.

[0113] It can be seen that, in order to better analyze the key event of the to-be-processed video, the key event analysis method provided by the embodiments of the present disclosure analyzes the target data stream collected in the same physical space as the to-be-processed video, so as to obtain the second time sequence feature. In this way, the to-be-processed video can be analyzed according to the second time sequence feature to determine the second video interval. Since there is a certain relationship between the first video interval and the second video interval, by performing feature processing on the video data of the first video interval and the second video interval, the same features between the two can be determined, and further the alarm event of the to-be-processed video can be analyzed according to the same features between the two.

[0114] In some implementable examples, in Figure 6 Based on the embodiments shown, S141 can be implemented by the following means: encoding the video data of the first video interval and the second video interval based on the pre-configured video encoding algorithm; obtaining the similar data between the video encoding data of the second video interval and the video encoding data of the first video interval to obtain the analysis result.

[0115] In some examples, in order to identify the similar data of the video data of the first video interval and the second video interval, the video data of the first video interval and the second video interval needs to be encoded, so that the similar data of the video data of the first video interval and the second video interval can be identified. For example, the video data of the first video interval and the second video interval is encoded by using a pre-configured video encoding algorithm, and by comparing the encoded video data of the first video interval and the encoded video data of the second video interval, the similar data of the video data of each second video interval and the video data of each first video interval can be determined.

[0116] It can be seen that, in order to better analyze the key event of the to-be-processed video, the key event analysis method provided by the embodiment of the disclosure can obtain the second time sequence feature by analyzing the target data stream collected in the same physical space as the to-be-processed video. In this way, the to-be-processed video can be analyzed according to the second time sequence feature to determine the second video interval. Further, the video data of the first video interval and the second video interval is encoded based on the pre-configured video encoding algorithm, so that the similar data of the video data of each second video interval and the video data of each first video interval can be identified, which avoids the situation that the similar data of the video data of each second video interval and the video data of each first video interval cannot be identified due to different encoding formats of the video data of the first video interval and the video data of the second video interval. After the similar data of the video data of each second video interval and the video data of each first video interval is obtained, the target video interval corresponding to the key event in the to-be-processed video can be determined according to the similar data, and the key event of the to-be-processed video can be analyzed.

[0117] In some implementable examples, the pre-configured video encoding algorithm is a FlowNet algorithm.

[0118] It can be seen that, in the key event analysis method provided by the embodiment of the disclosure, the video data of the first video interval and the second video interval is encoded by using the FlowNet algorithm, so that the gray value of each pixel in each image in the video data of the first video interval and the second video interval can be more accurately identified, and the similar data of the video data of the second video interval and the video data of the first video interval can be more accurately identified.

[0119] In some implementable examples, in the Figure 6 Based on the embodiment shown in the above, further, the S142 can be implemented by the following S1421 and S1422.

[0120] S1421, determining a third video interval based on the analysis result. The third video interval is a video interval in the second video interval, in which the total number of the similar data is greater than or equal to a preset threshold.

[0121] In some examples, the more similar data there is between the video data of the first and second video intervals, the more common features they share. This allows for more accurate correlation of relevant key events during analysis. Therefore, the second video interval can be determined based on the relationship between the total number of similar data points and a preset threshold.

[0122] S1422. Based on the first video interval and the third video interval, determine the target video interval corresponding to the key events in the video to be processed.

[0123] As can be seen, in order to better analyze the key events in the video to be processed, the key event analysis method provided in this disclosure analyzes the target data stream collected in the same physical space as the video to be processed, thereby obtaining a second temporal feature. Thus, based on the second temporal feature, the video to be processed can be analyzed to determine a second video interval. The more similar data there is between the video data of the first video interval and the video data of the second video interval, the more similar features they have. Therefore, based on the similarity data between the video data of the second video interval and the video data of the first video interval, the target video interval corresponding to the key events in the video to be processed can be determined, and the key events in the video to be processed can be analyzed.

[0124] In some feasible examples, the target data stream includes at least one of the following: audio data stream, video data stream, and sensing data stream; wherein the sensing data stream is acquired by a sensing device, which includes at least one of the following: acoustic sensor, ultrasonic sensor, electromagnetic wave sensor, infrared sensor, infrared camera, optical sensor, mechanical sensor, and temperature sensor.

[0125] For example, in combination Figure 1 The operational logic of the key event analysis method provided in this disclosure embodiment, as shown in scenario (a) and the example given in S122 above, is as follows: Figure 7The key event is an alarm event, and the to-be-processed video a is collected by the image collection device 2, and the target data stream d is collected by the sound collection device 3. The alarm event of the to-be-analyzed video collected by the image collection device 2 is marked to obtain the first video interval b. After the first video interval b is input into the computer system A, the first time sequence feature pool c corresponding to the first video interval b (the timestamp combination between 18:00-19:00 on April 26, 2022) can be obtained. Then, the computer system B determines the second encoding data e in the target data stream d that is aligned with the first time sequence feature pool c (the data in the target data stream corresponding to 18:00-19:00 on April 26, 2022) and the data in the target data stream d other than the second encoding data e (the data in the target data stream other than 18:00-19:00 on April 26, 2022, hereinafter referred to as target encoding data f) according to the first time sequence feature pool c, the target data stream d, and the digital encoding mode D corresponding to the target data stream d. Then, the second encoding data e is learned by the decision tree model E in the computer system C, so as to obtain a feature analysis model capable of extracting features with the same effective features as the second encoding data e. Then, by inputting the target encoding data f into the feature analysis model obtained by the computer system C, the third encoding data g with the same effective features as the second encoding data e is obtained (the target data stream between 14:00-15:00 on April 26, 2022 in the target data stream is determined to have the effective features of the target data stream between 18:00-19:00 on April 26, 2022 by the feature analysis model, so the target data stream between 14:00-15:00 on April 26, 2022 can be the third encoding data g). Further, the third encoding data g is processed by the computer system D, so as to obtain the second time sequence feature pool h corresponding to the third encoding data g (14:00-15:00 on April 26, 2022). In addition, the reference video k other than the to-be-processed video a is also provided in the image collection device 2, and at this time, the videos in the to-be-analyzed video a and the reference video k that are aligned with the second time sequence feature pool h are analyzed by the computer system E, so as to obtain the first candidate interval i (the data in the to-be-analyzed video a corresponding to 14:00-15:00 on April 26, 2022) and the second candidate interval j (the data in the reference video k corresponding to 14:00-15:00 on April 26, 2022), respectively. Then, the first candidate interval i, the second candidate interval j, and the first video interval b (for example, the data of 18:00-19:00 on April 26, 2022 in the to-be-analyzed video a) are encoded by the computer system F, so as to obtain the similar data of the first candidate interval i and the first video interval b, and the similar data of the second candidate interval j and the first video interval b.Finally, the computer system G analyzes the similarity data of the first candidate interval i and the first video interval b, and the similarity data of the second candidate interval j and the first video interval b, thereby analyzing the key event of the to-be-processed video.

[0126] It can be seen that, in order to better identify the key event of the to-be-processed video, the key event analysis method provided by the embodiment of the present disclosure can more accurately analyze the key event of the to-be-processed video by combining the time sequence characteristics of the target data stream collected in the same physical space and in different reporting manners when analyzing the key event of the to-be-processed video.

[0127] The above mainly introduces the scheme provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it contains the hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0128] The embodiment of the present application can divide the function modules of the server according to the above method examples. For example, each function module can be divided according to each function, or two or more functions can be integrated in one processing module. The integrated module can be realized in the form of hardware or software function module. It should be noted that the division of modules in the embodiment of the present application is illustrative, and is only a logical function division. Actual implementation can have another division method.

[0129] As shown in FIG. 1, Figure 8 The embodiment of the present application provides a structural schematic diagram of a server 1. The server 1 includes an acquisition unit 101 and a processing unit 102.

[0130] The acquisition unit 101 is configured to acquire a first time sequence feature pool of a to-be-processed video in a first video interval and a target data stream. The target data stream is collected in the same physical space as the to-be-processed video. The processing unit 102 is configured to determine a second time sequence feature pool in the target data stream acquired by the acquisition unit 101 based on the first time sequence feature pool acquired by the acquisition unit 101. The processing unit 102 is further configured to determine a second video interval based on the second time sequence feature pool. The processing unit 102 is further configured to determine a target video interval corresponding to a key event in the to-be-processed video based on the first video interval and the second video interval.

[0131] In some implementable examples, the processing unit 102 is specifically configured to digitally encode the target data stream acquired by the acquisition unit 101 to obtain first encoded data; the processing unit 102 is specifically configured to acquire, in the first encoded data, second encoded data aligned with the first pool of time-series features; the processing unit 102 is specifically configured to acquire, in the first encoded data, third encoded data having valid features in the second encoded data; and the processing unit 102 is specifically configured to acquire the second pool of time-series features corresponding to the third encoded data.

[0132] In some implementable examples, the processing unit 102 is specifically configured to learn the second encoded data by using a preset feature analysis model; the feature analysis model is used for making a classification decision on the distribution features of the encoded data in different frequency domains; and the processing unit 102 is specifically configured to process the first encoded data by using the learned feature analysis model to obtain the third encoded data.

[0133] In some implementable examples, the feature analysis model is a gradient boosting decision tree model.

[0134] In some implementable examples, the processing unit 102 is specifically configured to digitally encode the target data stream acquired by the acquisition unit 101 to obtain first encoded data; the processing unit 102 is specifically configured to acquire, in the first encoded data, second encoded data aligned with the first pool of time-series features; and the processing unit 102 is specifically configured to acquire the second pool of time-series features corresponding to the second encoded data.

[0135] In some implementable examples, the processing unit 102 is specifically configured to determine, based on the second pool of time-series features and a to-be-processed video acquired by the acquisition unit 101, a second video interval of the key event.

[0136] In some implementable examples, the second video interval includes a first candidate interval and a second candidate interval; the processing unit 102 is specifically configured to determine, based on the second pool of time-series features and the to-be-processed video acquired by the acquisition unit, the first candidate interval; and the processing unit 102 is specifically configured to determine, based on the second pool of time-series features and a reference video, the second candidate interval; wherein the reference video is historical video data collected by a video collection device of the to-be-processed video.

[0137] In some implementable examples, the processing unit 102 is specifically configured to perform feature similarity analysis on video data in the first video interval and the second video interval to obtain an analysis result; and the processing unit 102 is specifically configured to determine, based on the analysis result, a target video interval corresponding to the key event in the to-be-processed video.

[0138] In some embodiments, the processing unit 102 is specifically configured to encode the video data of the first video interval and the second video interval based on a pre-configured video encoding algorithm; and obtain similarity data of the video encoding data of the second video interval and the video encoding data of the first video interval to obtain an analysis result.

[0139] In some embodiments, the pre-configured video encoding algorithm is an optical flow network algorithm.

[0140] In some embodiments, the processing unit 102 is specifically configured to determine a third video interval based on the similarity data. The third video interval is a second video interval whose total number of similarity data is greater than or equal to a preset threshold. In some embodiments, the processing unit 102 is specifically configured to determine a target video interval corresponding to a key event in the to-be-processed video based on the first video interval and the third video interval.

[0141] In some embodiments, the target data stream includes at least one of an audio data stream, a video data stream and a sensing data stream. The sensing data stream is obtained by a sensing device, and the sensing device includes at least one of an acoustic sensor, an ultrasonic sensor, an electromagnetic wave sensor, an infrared sensor, an infrared camera, an optical sensor, a mechanical sensor and a temperature sensor.

[0142] All related contents of each step involved in the above method embodiments can be cited to the function description of the corresponding function module, and the functions will not be repeated here.

[0143] Of course, the server 1 provided by the embodiments of the present application includes but is not limited to the above-mentioned modules. For example, the server 1 can also include a storage unit 103. The storage unit 103 can be used to store the program code of the server 1, and can also be used to store the data generated by the server 1 during operation, such as the data in the write request.

[0144] Figure 9 A structural diagram of a server 1 provided by the embodiments of the present application is shown in FIG. 1. The server 1 can include at least one processor 51, a memory 52, a communication interface 53 and a communication bus 54. Figure 9

[0145] The specific introduction of each component of the server 1 will be described below. Figure 9

[0146] ​​The processor 51 is the control center of the server 1. It can be a single processor or a collective term for multiple processing elements. For example, the processor 51 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field-programmable gate arrays (FPGAs).

[0147] In a specific implementation, as one example, processor 51 may include one or more CPUs, for example... Figure 9 CPU0 and CPU1 are shown in the diagram. Furthermore, as one embodiment, server 1 may include multiple processors, for example... Figure 9 The processors 51 and 56 shown are illustrated. Each of these processors can be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0148] The memory 52 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0149] In a specific implementation, the memory 52 is configured to store data and software programs for implementing the present application. The processor 51 can execute various functions of the air conditioner by running or executing the software programs stored in the memory 52 and calling the data stored in the memory 52.

[0150] The communication interface 53 is configured to communicate with other devices or communication networks, such as a Radio Access Network (RAN), a Wireless Local Area Networks (WLAN), a terminal, a cloud, etc., using any transceiver-like device. The communication interface 53 can include an obtaining unit 101 to implement an obtaining function.

[0151] The communication bus 54 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 only one bus or one type of bus is represented by a thick line, but this does not indicate that there is only one bus or only one type of bus.

[0152] As an example, in combination with Figure 8 , the obtaining unit 101 in the server 1 implements the same function as the communication interface 53 in Figure 9 , the processing unit 102 implements the same function as the processor 51 in Figure 9 , and the storage unit 103 implements the same function as the memory 52 in Figure 9 .

[0153] Another embodiment of the present application also provides a computer readable storage medium, which stores instructions, when the instructions are run on a computer, causing the computer to execute the method shown in the above method embodiment.

[0154] In some embodiments, the disclosed method can be implemented as computer program instructions encoded in a computer readable storage medium in a machine readable format or encoded in other non-transitory media or articles.

[0155] Figure 10 A conceptual partial view of a computer program product provided by an embodiment of the present application is schematically shown, and the computer program product includes a computer program for executing a computer process on a computing device.

[0156] In one embodiment, a computer program product is provided using a signal bearing medium 410. The signal bearing medium 410 can include one or more program instructions implementing one or more embodiments of the above-described functionality or partial functionality. Thus, for example, one or more features of S11-S14 can be undertaken by one or more instructions associated with the signal bearing medium 410, with reference to the embodiments shown in Figure 2 The above-described functionality or partial functionality is described in the Figure 2 Thus, for example, with reference to the embodiments shown in Figure 10 The program instructions in the

[0157] In some examples, the signal bearing medium 410 can comprise a computer- readable medium 411, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electronically erasable programmable read-only memory (EEPROM), a floppy disk, a Zip® disk, a Blu-ray® disc, a flash memory, a magnetic tape, other types of magnetic media, optical media, solid state media, and / or other computer-readable media.

[0158] In some embodiments, the signal bearing medium 410 can comprise a computer- recordable medium 412, such as, but not limited to, a floppy disk, a Zip® disk, a Blu-ray® disc, a flash memory, a magnetic tape, other types of magnetic media, optical media, solid state media, and / or other computer-readable media.

[0159] In some embodiments, the signal bearing medium 410 can comprise a communication medium 413, such as, but not limited to, a digital and / or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, and / or the like).

[0160] The signal bearing medium 410 can be conveyed by a wireless form of the communication medium 413 (e.g., a wireless communication medium complying with the IEEE 802.41 standards or other transmission protocols). The one or more program instructions can be, for example, computer-executable instructions or logic-implementing instructions.

[0161] In some examples, such as for the write data device described in Figure 2 The write data device can be configured to provide various operations, functions, or actions in response to the one or more program instructions of the computer-readable medium 411, the computer-recordable medium 412, and / or the communication medium 413.

[0162] From the above description of embodiments, it is manifest that various techniques can be used by those skilled in the art of the described functions, and / or structures, and the actual details can also vary depending on implementation choices. It is therefore intended that the embodiments described herein be considered in all respects as illustrative and not restrictive, with the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are intended to be embraced therein.

[0163] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. For example, the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different apparatuses can be indirect couplings or communication connections through some interfaces, and there can be electric, mechanical or other forms.

[0164] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, i.e., can be located in one place, or can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0165] In addition, each functional unit in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically as separate units, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware, or in the form of software functional units.

[0166] When the integrated unit is implemented in the form of software functional units and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application essentially, or the part that contributes to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, and includes several instructions for causing an apparatus (which can be a single chip machine, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, and various other media that can store program codes.

[0167] The above description is merely a specific implementation of the present disclosure, and enables those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A critical incident analysis method, characterized in that, include: The first temporal feature pool of the video to be processed in the first video interval is obtained, as well as the target data stream; wherein the target data stream and the video to be processed are acquired in the same physical space. Based on the first temporal feature pool, a second temporal feature pool in the target data stream is determined; the determination of the second temporal feature pool in the target data stream based on the first temporal feature pool includes: digitally encoding the target data stream to obtain first encoded data; obtaining second encoded data aligned with the first temporal feature pool from the first encoded data; obtaining third encoded data from the first encoded data, the third encoded data possessing the effective features in the second encoded data; and obtaining the second temporal feature pool corresponding to the third encoded data. Based on the second temporal feature pool, the second video interval is determined; Based on the first video interval and the second video interval, the target video interval corresponding to the key event in the video to be processed is determined.

2. The critical event analysis method according to claim 1, characterized in that, The step of obtaining the third encoded data from the first encoded data includes: The second encoded data is learned using a preset feature analysis model; the feature analysis model is used to make classification decisions based on the distribution characteristics of the encoded data in different frequency domains. The first encoded data is processed using the learned feature analysis model to obtain the third encoded data.

3. The critical event analysis method according to claim 2, characterized in that, The feature analysis model is a gradient boosting decision tree model.

4. The critical event analysis method according to claim 1, characterized in that, The step of determining the second video interval based on the second temporal feature pool includes: Based on the second temporal feature pool and the video to be processed, the second video interval of the key event is determined.

5. The critical event analysis method according to claim 1, characterized in that, The second video interval includes the first candidate interval and the second candidate interval; The step of determining the second video interval based on the second temporal feature pool includes: Based on the second temporal feature pool and the video to be processed, a first candidate interval is determined; Based on the second temporal feature pool and the reference video, a second candidate interval is determined; wherein, the reference video is historical video data collected by the video acquisition device of the video to be processed.

6. The critical event analysis method according to claim 1, characterized in that, The step of determining the target video interval corresponding to the key event in the video to be processed based on the first video interval and the second video interval includes: Perform feature similarity analysis on the video data of the first video interval and the second video interval, and determine the analysis results; Based on the analysis results, the target video intervals corresponding to the key events in the video to be processed are determined.

7. The critical event analysis method according to claim 6, characterized in that, The step of performing feature similarity analysis on the video data of the first video interval and the second video interval, and determining the analysis results, includes: The video data of the first video interval and the second video interval are encoded based on a pre-configured video encoding algorithm; The analysis results are obtained by acquiring similar data between the video encoded data of the second video interval and the video encoded data of the first video interval.

8. The critical event analysis method according to claim 7, characterized in that, The pre-configured video encoding algorithm is an optical flow network algorithm.

9. The critical event analysis method according to claim 7, characterized in that, The step of determining the target video interval corresponding to the key events in the video to be processed based on the analysis results includes: Based on the similar data, a third video interval is determined, which is a second video interval in which the total number of similar data is greater than or equal to a preset threshold. Based on the first video interval and the third video interval, the target video interval corresponding to the key event in the video to be processed is determined.

10. The critical event analysis method according to any one of claims 1-5, characterized in that, The target data stream includes at least one of the following: audio data stream, video data stream, and sensor data stream; wherein the sensor data stream is acquired by a sensing device, and the sensing device includes at least one of the following: acoustic sensor, ultrasonic sensor, electromagnetic wave sensor, infrared sensor, infrared camera, optical sensor, mechanical sensor, and temperature sensor.

11. A critical event analysis device, characterized in that, include: An acquisition unit is used to acquire a first temporal feature pool of the video to be processed in a first video interval, and a target data stream; wherein the target data stream and the video to be processed are acquired in the same physical space; A processing unit is configured to determine a second temporal feature pool in the target data stream acquired by the acquisition unit based on a first temporal feature pool acquired by the acquisition unit; the step of determining the second temporal feature pool in the target data stream based on the first temporal feature pool includes: digitally encoding the target data stream to obtain first encoded data; acquiring second encoded data aligned with the first temporal feature pool from the first encoded data; acquiring third encoded data from the first encoded data, the third encoded data possessing valid features from the second encoded data; and acquiring a second temporal feature pool corresponding to the third encoded data. The processing unit is further configured to determine a second video interval based on the second temporal feature pool; The processing unit is further configured to determine the target video interval corresponding to the key event in the video to be processed based on the first video interval and the second video interval.

12. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to cause the electronic device to implement the critical event analysis method according to any one of claims 1-10 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computing device, causes the computing device to implement the critical event analysis method according to any one of claims 1-10.

Images