AI video digital analysis method and system

Abstract

The present invention discloses an AI video digital analysis method and system: real-time acquisition of video data of a monitoring area, and sending the video data to a monitoring platform; preprocessing the video data to obtain the video data to be analyzed; segmenting the video data to be analyzed, extracting all video frames in any video segment, and assigning all video frames to a first module and a second module according to a preset ratio, the first module analyzing the assigned first video frame to obtain a first detection result, the second module analyzing all assigned second video frames to obtain a second detection result corresponding to each second video frame; the first module obtaining the second detection result of the second module, and adjusting the second detection result based on the first detection result; after traversing all video segments, outputting the monitoring target and / or the motion trajectory of the monitoring target in the video data. The technical solution of the present invention can reduce the amount of calculation while ensuring the detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of video analysis, and specifically relates to an AI video digital analysis method and system. Background Art

[0002] Traditional video surveillance systems use CCTV cameras installed in each major facility area, store real-time video data on servers, and display live video screens or recorded playbacks, providing control room controllers with multiple screen experiences and continuous operations. In recent years, with the rapid development of artificial intelligence technology, people are trying to apply artificial intelligence technology to the field of video surveillance, and intelligent analysis technology is an important part of video surveillance systems.

[0003] In the prior art, for example, Chinese patent CN117036352B discloses a video analysis method and system based on artificial intelligence, which obtains multiple video analysis tasks, extracts the initial surveillance video shot by the corresponding camera in the video analysis task, pre-processes the initial surveillance video, and obtains the video to be analyzed; obtains the tracking target in the video analysis task, identifies and processes the video to be analyzed based on the tracking target, and determines the target analysis object; segments the image data to be processed containing the target analysis object from the video to be analyzed; inputs the image data to be processed into the task analysis model, and performs behavior analysis on the target analysis object in the image data to be processed through the task analysis model, and the task analysis model outputs the analysis result corresponding to the video analysis task. Another example is Chinese patent application CN111797756A, which discloses a video analysis method, device and medium based on artificial intelligence, which, when receiving the video analysis, obtains the original video collected by the acquisition device; splits the original video to obtain at least one video segment; extracts features from each video segment using multiple pre-trained models, and outputs at least one feature of each video segment; analyzes at least one feature of each video segment to obtain analysis data.

[0004] If a model with high analysis accuracy is used in the video analysis model in the above-mentioned prior art, the analysis model needs to occupy a large amount of memory and has a large amount of calculation. If a model with low analysis accuracy is used, the analysis result will be inaccurate. Therefore, it is an urgent problem to provide an AI video digital analysis method and system to save computing resources, reduce the amount of calculation, and improve the accuracy of analysis. Summary of the invention

[0005] In response to the above-mentioned technical problems, the present invention provides an AI video digital analysis method and system.

[0006] In a first aspect, the present invention provides an AI video digital analysis method, the method comprising:

[0007] Step 1: The video acquisition device collects video data of the monitoring area in real time and sends the video data to the monitoring platform;

[0008] Step 2: preprocess the video data to obtain the video data to be analyzed;

[0009] Step 3, segmenting the video data to be analyzed, extracting all video frames in any video segment, and allocating all video frames to the first module and the second module according to a preset ratio, wherein the analysis accuracy of the first module is higher than that of the second module;

[0010] Step 4: The first module analyzes the allocated first video frame to obtain a first detection result, and the second module analyzes all allocated second video frames to obtain a second detection result corresponding to each second video frame;

[0011] Step 5: The first module obtains the second detection result of the second module, and adjusts the second detection result based on the first detection result;

[0012] Step 6: After traversing all video segments, output the monitoring target and / or the motion trajectory of the monitoring target in the video data.

[0013] Specifically, in step 1, the video acquisition device sends video data to the monitoring platform, including:

[0014] Step 111: compress and encode the video data to generate video data to be transmitted, and divide the transmitted video data to generate multiple transmission data packets;

[0015] Step 112: Pack N1 transmission data packets to generate a transmission array, and set a number and an array identifier for each transmission array in turn, wherein the number is the sequence number of the transmission array in the transmission code stream, and the array identifier indicates whether the array is a transmission array or a repeating array;

[0016] Step 113, generating a repeating array for each transmission array, and setting an array identifier different from the transmission array corresponding to any repeating array and a number that is the same as the transmission data packet corresponding to any repeating array for any repeating array, wherein the data packet in any repeating array is the same transmission data packet as the transmission array corresponding to any repeating array;

[0017] Step 114, obtaining the current repetition degree, adding N2 arbitrary repetition arrays corresponding to the repetition degree to the end of the transmission array corresponding to any repetition array to generate the first video data;

[0018] Step 115: Generate a transmission code stream based on all the first video data and send it to the monitoring platform.

[0019] Specifically, step 1 includes:

[0020] Step 121, extracting the array identifier and number of any array, determining whether the array identifier and number of any array are the same as the array identifier and number of the previously received array, if so, deleting any array, if not, proceeding to step 122, wherein any array is a transmission array or a repeating array;

[0021] Step 122, determine whether the number of any array is continuous with the previously saved number, if so, decompose any array into N1 data packets and save the number of any array, if not, proceed to step 123;

[0022] Step 123, determine whether the number of any array is the same as the previously saved number, if the same, delete any array, if not the same, determine that the array between the number of any array and the previously saved number is lost, and feed back the number corresponding to the lost array to the video acquisition device;

[0023] Step 124: after receiving all transmission data packets, transmission video data is generated, and the transmission video data is decompressed to obtain video data.

[0024] Specifically, in step 114, the calculation method of the repeatability is:

[0025] The monitoring platform calculates the array loss rate based on the number of array losses within the first preset time period and feeds it back to the video acquisition device;

[0026] The video acquisition device calculates the repetition based on the array loss rate, and the calculation formula is:

[0027] R = [α × L]

[0028] Among them, R is the repetition rate, L is the array loss rate, and α is the parameter weight coefficient.

[0029] Specifically, preprocessing includes:

[0030] Step 21: Sampling the video data according to a second preset time period to obtain sampled image data;

[0031] Step 22: Analyze the sampled image data to obtain the value corresponding to each adjustment parameter;

[0032] Step 23: Compare the value of any adjustment parameter with the standard range of any adjustment parameter. When the value exceeds the standard range, use the adjustment module corresponding to any adjustment parameter to adjust the video data.

[0033] Step 24: After the adjustment is completed, the video data to be analyzed is output.

[0034] Specifically, in step 4, the first detection result includes an object identifier of each first target to be analyzed in the first video frame, and vertex coordinates of a circumscribed rectangle of each first target to be analyzed;

[0035] The second detection result includes displacement information, an object identifier, and vertex coordinates of a circumscribed rectangle of each second target to be analyzed in any second video frame.

[0036] Specifically, step 5 includes:

[0037] Step 51, defining the first video frame as a reference frame, and defining the second video frame captured at a next moment after the reference frame is captured as a video frame to be adjusted;

[0038] Step 52, obtaining the bounding rectangle of the j-th first target to be analyzed on the reference frame, and determining whether there is a target to be analyzed on the video frame to be adjusted. If yes, proceed to step 53; if not, proceed to step 55, wherein the target to be analyzed is the second target to be analyzed corresponding to the j-th first target to be analyzed, j is a positive integer from 1 to J, and J is the total number of first targets to be analyzed in the reference frame;

[0039] Step 53, obtaining the bounding rectangle of the target to be analyzed to be adjusted, calculating the intersection and union ratio of the bounding rectangle of the j-th first target to be analyzed and the bounding rectangle of the target to be analyzed to be adjusted, if the intersection and union ratio is greater than or equal to a preset value, proceeding to step 54, if the intersection and union ratio is less than the preset value, proceeding to step 55;

[0040] Step 54, delete the analysis target to be adjusted, adjust the position of the jth first analysis target to be analyzed based on the displacement information of the analysis target to be adjusted to obtain the adjusted analysis target, if j<J, set j=j+1, return to step 52, otherwise go to step 56;

[0041] Step 55, delete the jth first target to be analyzed, if j<J, set j=j+1, return to step 52, otherwise go to step 56;

[0042] Step 56: Generate an adjusted video frame based on all adjusted analysis targets and the third target to be analyzed, and determine whether the adjustment of all second video frames is completed. If so, output the adjusted video segment; if not, define the adjusted video frame as the reference frame, and return to step 51, wherein the third target to be analyzed is the second target to be analyzed that has not been deleted in the analysis target to be adjusted.

[0043] In a second aspect, the present invention further provides an AI video digital analysis system, the system comprising: a video acquisition device and a monitoring platform, the monitoring platform comprising a preprocessing module, a distribution module, a first module, a second module and a result output module;

[0044] Video acquisition equipment, used to collect video data of the monitoring area in real time and send the video data to the monitoring platform;

[0045] A preprocessing module is used to preprocess the video data to obtain the video data to be analyzed;

[0046] an allocation module, used for segmenting the video data to be analyzed, extracting all video frames in any video segment, and allocating all video frames to the first module and the second module according to a preset ratio, wherein the analysis accuracy of the first module is higher than that of the second module;

[0047] The second module is used to analyze all the allocated second video frames and obtain a second detection result corresponding to each second video frame;

[0048] The first module is used to analyze the allocated first video frame to obtain a first detection result, and is also used to obtain a second detection result of the second module, and adjust the second detection result based on the first detection result;

[0049] The result output module is used to output the monitoring target and / or the motion trajectory of the monitoring target in the video data after traversing all video segments.

[0050] The present invention discloses an AI video digital analysis method and system. When analyzing video data to be analyzed, the video data to be analyzed is first segmented, and then all video frames in any video segment are extracted, and all video frames are allocated to a first module with high analysis accuracy and a second module with low analysis accuracy according to a preset ratio. The first module performs high-precision analysis on the first video frame to obtain a first detection result, and the second module performs low-precision analysis on the second video frame to obtain a second detection result. Finally, the second detection result is adjusted based on the first detection result to improve the accuracy of the second detection result, and all video segments are traversed. Based on the first detection result and the adjusted second detection result, the monitoring target and / or the motion trajectory of the monitoring target in the video data are output. According to the technical solution of the present invention, a part of the video frames is analyzed with high precision, and another part of the video frames is analyzed with low precision, and then the low-precision analysis result is adjusted based on the high-precision analysis result. While ensuring the accuracy of the analysis result, it is not necessary to perform high-precision analysis on all video frames, thereby reducing the amount of calculation and saving computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 It is a flow chart of the AI ​​video digital analysis method of the present invention;

[0053] Figure 2 A schematic diagram of a transmission code stream according to an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of adjusting the target to be analyzed according to an embodiment of the present invention;

[0055] Figure 4 It is a structural schematic diagram of the AI ​​video digital analysis system of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the specific embodiments described herein are only used to explain the present invention and are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0057] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0058] Figure 1 FIG. 1 is a flow chart of an embodiment of the AI ​​video digital analysis method provided by the present invention, and the flow chart specifically includes:

[0059] Step 1: The video acquisition device collects video data of the monitoring area in real time and sends the video data to the monitoring platform.

[0060] The video acquisition device may include at least one camera device, which records real-time video in the monitoring area, and each camera device is installed in a fixed position.

[0061] Specifically, in step 1, the video acquisition device sends video data to the monitoring platform, including:

[0062] Step 111: compress and encode the video data to generate video data to be transmitted, and divide the transmitted video data to generate multiple transmission data packets.

[0063] Step 112: Pack N1 transmission data packets to generate a transmission array, and set a number and an array identifier for each transmission array in turn, where the number is the sequence number of the transmission array in the transmission code stream, and the array identifier indicates whether the array is a transmission array or a repeating array.

[0064] Step 113: Generate a repeating array for each transmission array, and set an array identifier different from the transmission array corresponding to any repeating array and a number that is the same as the transmission data packet corresponding to any repeating array for any repeating array, wherein the data packet in any repeating array is the same transmission data packet as the transmission array corresponding to any repeating array.

[0065] Step 114: Obtain the current repetition degree, and add N2 arbitrary repetition arrays corresponding to the repetition degree to the end of the transmission array corresponding to any repetition array to generate the first video data.

[0066] Step 115: Generate a transmission code stream based on all the first video data and send it to the monitoring platform.

[0067] Exemplarily, the transmission data packet is a PS packet or a TS packet.

[0068] The embodiment of the present invention is described with the number of transmission data packets being 6, N1 being 2, and N2 being 1. Two transmission data packets are packaged to generate a transmission array, and three transmission arrays are generated. The first transmission array is numbered 1 and array identifier 1, the second transmission array is numbered 2 and array identifier 1, and the third transmission array is numbered 3 and array identifier 1. When sending, a first repeating array is added after the first transmission array, and the first repeating array is numbered 1 and the array identifier is 2. A second repeating array is added after the second transmission array, and the second repeating array is numbered 2 and the array identifier is 2. A third repeating array is added after the third transmission array, and the third repeating array is numbered 3 and the array identifier is 2.

[0069] According to the technical solution of the present invention, if the transmission array is lost during the transmission process, a repeated array can be used for error correction and correction, thereby improving the error tolerance of video transmission; different array identifiers are set for the repeated array and the transmission array to facilitate distinction, and after the transmission array is received, the redundant repeated array is quickly deleted; at the same time, compared with the method of adding error correction codes, the technical solution of the present invention only needs to copy the array with the same valid content after the transmission array, and the calculation time is less.

[0070] Specifically, step 1 includes:

[0071] Step 121, extract the array identifier and number of any array, determine whether the array identifier and number of any array are the same as the array identifier and number of the previously received array, if so, delete any array, if not, enter step 122, where any array is a transmission array or a repeated array.

[0072] Step 122, determine whether the number of any array is continuous with the previously saved number. If so, decompose any array into N1 data packets and save the number of any array. If not, go to step 123.

[0073] Step 123, determine whether the number of any array is the same as the previously saved number. If they are the same, delete any array. If they are not the same, determine that the array between the number of any array and the previously saved number is lost, and feed back the number corresponding to the lost array to the video acquisition device.

[0074] Step 124: after receiving all transmission data packets, transmission video data is generated, and the transmission video data is decompressed to obtain video data.

[0075] by Figure 2 Take as an example to illustrate the embodiment of the present invention, and for the convenience of explanation, a "receiving sequence number" is added. If the third array is received, and the previous received array is the second array, the array identifier and number of the received array are the same as the array identifier and number of the previous received array, indicating that the array is a repeated received array, and the array is deleted. If the fourth array is received, and the previous saved array is the first array, the array identifiers of the two are the same, and the numbers are continuous, indicating that the array is received for the first time and is a valid array, and the array is saved. If the seventh array is received, and the previous saved array is the first array, the array identifiers of the two are the same, and the numbers are discontinuous, indicating that the array numbered 2 is lost, and the number corresponding to the lost array is fed back to the video acquisition device to resend the lost part of the array.

[0076] Specifically, in step 114, the calculation method of the repeatability is:

[0077] The monitoring platform calculates the array loss rate based on the number of array losses within the first preset time period and feeds it back to the video acquisition device;

[0078] The video acquisition device calculates the repetition based on the array loss rate, and the calculation formula is:

[0079] R = [α × L]

[0080] Among them, R is the repetition rate, L is the array loss rate, and α is the parameter weight coefficient.

[0081] The first preset time period is set according to the experience of those skilled in the art or according to actual application scenarios, and the embodiments of the present application are not limited to this.

[0082] In the repetition calculation formula, [] is a rounding operator, and the integer part of the product of the array loss rate and the parameter weight coefficient is used as the repetition of adding repeated arrays. The repetition is calculated based on the array loss rate. The more array losses there are, the worse the network stability is. A higher repetition is set to improve the error correction capability.

[0083] Step 2: Preprocess the video data to obtain the video data to be analyzed.

[0084] Due to external factors such as weather, the video data collected by the video acquisition device will be affected. In order to improve the accuracy of video analysis, the video data is pre-processed after it is received.

[0085] Specifically, preprocessing includes:

[0086] Step 21: Sample the video data according to a second preset time period to obtain sampled image data.

[0087] Step 22: Analyze the sampled image data and obtain the value corresponding to each adjustment parameter.

[0088] Step 23: Compare the value of any adjustment parameter with the standard range of any adjustment parameter. When the value exceeds the standard range, use the adjustment module corresponding to any adjustment parameter to adjust the video data.

[0089] Step 24: After the adjustment is completed, the video data to be analyzed is output.

[0090] The influence of external factors is phased, and in order to reduce the amount of calculation, the video data is periodically evaluated and processed according to the evaluation results. The second preset time period is set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiment of the present application does not limit this.

[0091] Exemplarily, the adjustment parameters include ghosting, jitter, brightness, etc.

[0092] The sampled image data is analyzed to obtain the value corresponding to each adjustment parameter, and the video data is processed based on the value of the adjustment parameter. For example, if the ghosting degree is 0.7, which exceeds the preset value of 0.4, the video data is input into the adjustment module corresponding to the ghosting for adjustment.

[0093] According to the technical solution of the present invention, the video data can be pre-processed in a targeted manner, thereby improving the video processing efficiency and saving computing resources while ensuring the video picture effect.

[0094] Step 3: Segment the video data to be analyzed, extract all video frames in any video segment, and distribute all video frames to the first module and the second module according to a preset ratio, wherein the analysis accuracy of the first module is higher than that of the second module.

[0095] The preset ratio is set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiments of the present application do not limit this. For example, the preset ratio is 1:4, the first frame of the video is allocated to the first module, and the second, third, fourth, and fifth frames of the video are allocated to the second module.

[0096] Preferably, the first module and the second module are located on different devices. The second module is located on the video receiving device, the first module is located on a second device other than the video receiving device, and the second device communicates with the video receiving device wirelessly or by wire.

[0097] Due to factors such as memory, the video receiving device cannot have sufficient computing resources. In order not to affect the operation of the device, analysis modules with lower analysis accuracy are set, such as Nano Det model, Micro-YOLO model, HED model, etc., to reduce the amount of calculation while identifying objects in the video frame.

[0098] Step 4: The first module analyzes the allocated first video frame to obtain a first detection result, and the second module analyzes all allocated second video frames to obtain a second detection result corresponding to each second video frame.

[0099] Specifically, in step 4, the first detection result includes the object identification of each first target to be analyzed in the first video frame, and the vertex coordinates of the circumscribed rectangle of each first target to be analyzed;

[0100] The second detection result includes displacement information of each second target to be analyzed in any second video frame, an object identifier, and vertex coordinates of a circumscribed rectangle of each second target to be analyzed.

[0101] Exemplarily, the objects are identified as people, cars, electric vehicles, electric poles, bus stops, traffic lights, etc.

[0102] Preferably, the target to be analyzed is a pre-set monitoring target.

[0103] Preferably, the first video frame assigned to the first module is stored in the second module, and the first video frame is analyzed to obtain each second target to be analyzed in the first video frame, so as to calculate the displacement information of each second target to be analyzed in the second video frame immediately following the first video frame.

[0104] Step 5: The first module obtains the second detection result of the second module, and adjusts the second detection result based on the first detection result.

[0105] Specifically, step 5 includes:

[0106] Step 51: define the first video frame as a reference frame, and define the second video frame captured at a next moment after the reference frame is captured as a video frame to be adjusted.

[0107] Step 52, obtain the bounding rectangle of the j-th first target to be analyzed on the reference frame, and determine whether there is a target to be adjusted on the video frame to be adjusted. If yes, proceed to step 53; if not, proceed to step 55, wherein the target to be adjusted is the second target to be analyzed corresponding to the j-th first target to be analyzed, j is a positive integer from 1 to J, and J is the total number of first targets to be analyzed in the reference frame.

[0108] Step 53, obtain the bounding rectangle of the analysis target to be adjusted, calculate the intersection and union ratio of the bounding rectangle of the jth first analysis target and the bounding rectangle of the analysis target to be adjusted, if the intersection and union ratio is greater than or equal to the preset value, proceed to step 54, if the intersection and union ratio is less than the preset value, proceed to step 55.

[0109] Step 54 , delete the analysis target to be adjusted, adjust the position of the jth first analysis target to be analyzed based on the displacement information of the analysis target to be adjusted to obtain the adjusted analysis target, if j<J, set j=j+1, return to step 52 , otherwise go to step 56 .

[0110] Step 55 , delete the jth first target to be analyzed. If j<J, set j=j+1 and return to step 52 . Otherwise, go to step 56 .

[0111] Step 56: Generate an adjusted video frame based on all adjusted analysis targets and the third target to be analyzed, and determine whether the adjustment of all second video frames is completed. If so, output the adjusted video segment; if not, define the adjusted video frame as the reference frame, and return to step 51, wherein the third target to be analyzed is the second target to be analyzed that has not been deleted in the analysis target to be adjusted.

[0112] The preset value is set according to the experience of those skilled in the art or according to the actual application scenario, and the embodiments of the present application are not limited to this.

[0113] by Figure 3 The embodiment of the present invention is described by taking the example of Figure 3(a) is the reference frame, (b) is the video frame to be adjusted, (c) is the adjustment process, and (d) is the adjusted video frame. Draw the bounding rectangle of each first target to be analyzed on the reference frame, and draw the bounding rectangle of the second target to be analyzed on the reference frame based on the vertex coordinates of the bounding rectangle of each second target to be analyzed in the video frame to be adjusted. Get the bounding rectangle of the first target to be analyzed 1 on the reference frame, and there is an adjusted analysis target (second target to be analyzed 3) corresponding to the first target to be analyzed 1 in the video frame to be adjusted. The intersection-and-union ratio of the first target to be analyzed 1 and the second target to be analyzed 3 (i.e., the ratio of the area of ​​the overlapping part (gray shaded part) to the area of ​​the union of the first target to be analyzed 1 and the second target to be analyzed 3) is greater than the preset value, indicating that the first target to be analyzed 1 and the second target to be analyzed 3 are the same object, then delete the second target to be analyzed 3, and move the first target to be analyzed 1 based on the displacement information of the second target to be analyzed 3 to generate the adjusted analysis target 5. The bounding rectangle of the first target to be analyzed 2 on the reference frame is obtained. If there is no target to be analyzed 1 corresponding to the first target to be analyzed in the video frame to be adjusted, and the video frame to be adjusted is a frame after the reference frame, it means that the first target to be analyzed 2 has moved out of the camera lens at the time of acquisition of the video frame to be adjusted, then the second target to be analyzed 2 is deleted. There is an unadjusted second target to be analyzed 4 (third target to be analyzed 4) in the video to be adjusted, which is a new target to be analyzed in the video to be adjusted. The adjusted video frame (d) is generated based on the adjusted analysis target 5 and the third target to be analyzed 4.

[0114] The first model is a high-precision detection model with high target recognition accuracy and precision. The second model is a low-precision detection model. The second detection result of the second model is sent to the first model, and the second detection result of the second model is adjusted based on the reference frame to improve the overall detection accuracy.

[0115] According to the technical solution of the present invention, the second model only sends the detection result of the second target to be analyzed to the first model, the amount of data sent is small, and the second detection result of the second model is adjusted according to the first detection result of the first model, thereby reducing the amount of calculation while ensuring the accuracy of video analysis.

[0116] Step 6: After traversing all video segments, output the monitoring target and / or the motion trajectory of the monitoring target in the video data.

[0117] Figure 4 FIG. 1 is a schematic diagram of the structure of an embodiment of the AI ​​video digital analysis system provided by the present invention. Figure 4 As shown, the system includes: a video acquisition device 100 and a monitoring platform 200 , wherein the monitoring platform 200 includes a pre-processing module 201 , an allocation module 202 , a first module 203 , a second module 204 and a result output module 205 .

[0118] The video acquisition device 100 is used to acquire video data of the monitoring area in real time and send the video data to the monitoring platform 200 .

[0119] The preprocessing module 201 is used to preprocess the video data to obtain the video data to be analyzed.

[0120] The allocation module 202 is used to segment the video data to be analyzed, extract all video frames in any video segment, and allocate all video frames to the first module 203 and the second module 204 according to a preset ratio, wherein the analysis accuracy of the first module 203 is higher than that of the second module 204.

[0121] The second module 204 is used to analyze all the allocated second video frames to obtain a second detection result corresponding to each second video frame.

[0122] The first module 203 is used to analyze the allocated first video frame to obtain a first detection result, and is also used to obtain a second detection result of the second module 204, and adjust the second detection result based on the first detection result.

[0123] The result output module 205 is used to output the monitoring target and / or the motion trajectory of the monitoring target in the video data after traversing all video segments.

[0124] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0125] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0126] The above-mentioned embodiments only express the preferred implementation mode of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. AI video digital analysis method, characterized in that: The steps include: Step 1: The video acquisition device acquires video data of the monitoring area in real time and sends the video data to the monitoring platform; Step 2: preprocess the video data to obtain the video data to be analyzed; Step 3, segmenting the video data to be analyzed, extracting all video frames in any video segment, and allocating all the video frames to the first module and the second module according to a preset ratio, wherein the analysis accuracy of the first module is higher than that of the second module; Step 4: the first module analyzes the allocated first video frame to obtain a first detection result, and the second module analyzes all allocated second video frames to obtain a second detection result corresponding to each second video frame; Step 5: The first module obtains the second detection result of the second module, and adjusts the second detection result based on the first detection result; Step 6: After traversing all video segments, output the monitoring target in the video data and / or the motion trajectory of the monitoring target; The step 5 comprises: Step 51, defining the first video frame as a reference frame, and defining a second video frame captured at a next moment after the reference frame is captured as a video frame to be adjusted; Step 52, obtaining the bounding rectangle of the j-th first target to be analyzed on the reference frame, and determining whether there is a target to be analyzed on the video frame to be adjusted, if yes, proceeding to step 53, if not, proceeding to step 55, wherein the target to be analyzed is a second target to be analyzed corresponding to the j-th first target to be analyzed, j is a positive integer from 1 to J, and J is the total number of first targets to be analyzed in the reference frame; Step 53, obtaining the bounding rectangle of the target to be analyzed to be adjusted, calculating the intersection and union ratio of the bounding rectangle of the j-th first target to be analyzed and the bounding rectangle of the target to be analyzed to be adjusted, if the intersection and union ratio is greater than or equal to a preset value, proceeding to step 54, if the intersection and union ratio is less than the preset value, proceeding to step 55; Step 54, deleting the analysis target to be adjusted, adjusting the position of the j-th first analysis target to be analyzed based on the displacement information of the analysis target to be adjusted to obtain an adjusted analysis target, if j<J, setting j=j+1, and returning to step 52, otherwise proceeding to step 56; Step 55, delete the jth first target to be analyzed, if j<J, set j=j+1, return to the step 52, otherwise go to step 56; Step 56: Generate an adjusted video frame based on all adjusted analysis targets and the third target to be analyzed, determine whether the adjustment of all the second video frames is completed, and if so, output the adjusted video segment; if not, define the adjusted video frame as the reference frame, and return to step 51, wherein the third target to be analyzed is the second target to be analyzed that has not been deleted from the analysis target to be adjusted.

2. The AI ​​video digital analysis method according to claim 1, characterized in that: In step 1, the video acquisition device sends the video data to the monitoring platform, including: Step 111: compress and encode the video data to generate video data to be transmitted, and divide the transmitted video data to generate multiple transmission data packets; Step 112: Pack N1 transmission data packets to generate a transmission array, and set a number and an array identifier for each transmission array in turn, wherein the number is the sequence number of the transmission array in the transmission code stream, and the array identifier indicates whether the array is a transmission array or a repeating array; Step 113, generating the repeating array for each of the transmission arrays, and setting an array identifier different from the transmission array corresponding to any of the repeating arrays and a number that is the same as the transmission data packet corresponding to any of the repeating arrays for any of the repeating arrays, wherein the data packet in any of the repeating arrays is the same transmission data packet as the transmission array corresponding to any of the repeating arrays; Step 114, obtaining the current repetition degree, and adding any N2 repetition arrays corresponding to the repetition degree to the end of the transmission array corresponding to any repetition array to generate the first video data; Step 115: Generate a transmission code stream based on all the first video data and send it to the monitoring platform.

3. The AI ​​video digital analysis method according to claim 2, characterized in that: The step 1 then includes: Step 121, extracting the array identifier and number of any array, determining whether the array identifier and number of any array are the same as the array identifier and number of the previously received array, if so, deleting any array, if not, proceeding to step 122, wherein any array is the transmission array or the repeating array; Step 122, determine whether the number of any of the arrays is continuous with the previously saved number, if so, decompose any of the arrays into N1 data packets and save the number of any of the arrays, if not, proceed to step 123; Step 123, determining whether the number of any of the arrays is the same as the previously saved number, if the numbers are the same, deleting any of the arrays, if not the same, determining that the array between the number of any of the arrays and the previously saved number is lost, and feeding back the number corresponding to the lost array to the video acquisition device; Step 124: after receiving all transmission data packets, the transmission video data is generated, and the transmission video data is decompressed to obtain the video data.

4. The AI ​​video digital analysis method according to claim 3, characterized in that: In step 114, the calculation method of the repeatability is: The monitoring platform calculates the array loss rate based on the number of array losses within a first preset time period, and feeds back the result to the video acquisition device; The video acquisition device calculates the repetition based on the array loss rate, and the calculation formula is: R = [α × L] Wherein, R is the repetition degree, L is the array loss rate, and α is the parameter weight coefficient.

5. The AI ​​video digital analysis method according to claim 1, characterized in that: In the step 2, the pre-processing includes: Step 21: sampling the video data according to a second preset time period to obtain sampled image data; Step 22: Analyze the sampled image data to obtain the value corresponding to each adjustment parameter; Step 23: Compare the value of any adjustment parameter with the standard range of any adjustment parameter. When the value exceeds the standard range, use the adjustment module corresponding to any adjustment parameter to adjust the video data. Step 24: After the adjustment is completed, the video data to be analyzed is output.

6. The AI ​​video digital analysis method according to claim 1, characterized in that: In step 4, the first detection result includes the object identification of each first target to be analyzed in the first video frame, and the vertex coordinates of the circumscribed rectangle of each first target to be analyzed; The second detection result includes displacement information, an object identifier, and vertex coordinates of a circumscribed rectangle of each second target to be analyzed in any second video frame.

7. An AI video digital analysis system, used to implement the AI ​​video digital analysis method according to any one of claims 1 to 6, characterized in that: include: A video acquisition device and a monitoring platform, wherein the monitoring platform comprises a preprocessing module, an allocation module, a first module, a second module and a result output module; The video acquisition device is used to acquire video data of the monitoring area in real time and send the video data to the monitoring platform; The preprocessing module is used to preprocess the video data to obtain the video data to be analyzed; The allocation module is used to segment the video data to be analyzed, extract all video frames in any video segment, and allocate all the video frames to the first module and the second module according to a preset ratio, wherein the analysis accuracy of the first module is higher than that of the second module; The second module is used to analyze all the allocated second video frames to obtain a second detection result corresponding to each second video frame; The first module is used to analyze the allocated first video frame to obtain a first detection result, and is also used to obtain the second detection result of the second module, and adjust the second detection result based on the first detection result; The result output module is used to output the monitoring target and / or the motion trajectory of the monitoring target in the video data after traversing all video segments; The first module acquires the second detection result of the second module, and adjusting the second detection result based on the first detection result includes: Step 51, defining the first video frame as a reference frame, and defining a second video frame captured at a next moment after the reference frame is captured as a video frame to be adjusted; Step 52, obtaining the bounding rectangle of the j-th first target to be analyzed on the reference frame, and determining whether there is a target to be analyzed on the video frame to be adjusted, if yes, proceeding to step 53, if not, proceeding to step 55, wherein the target to be analyzed is a second target to be analyzed corresponding to the j-th first target to be analyzed, j is a positive integer from 1 to J, and J is the total number of first targets to be analyzed in the reference frame; Step 53, obtaining the bounding rectangle of the target to be analyzed to be adjusted, calculating the intersection and union ratio of the bounding rectangle of the j-th first target to be analyzed and the bounding rectangle of the target to be analyzed to be adjusted, if the intersection and union ratio is greater than or equal to a preset value, proceeding to step 54, if the intersection and union ratio is less than the preset value, proceeding to step 55; Step 54, deleting the analysis target to be adjusted, adjusting the position of the j-th first analysis target to be analyzed based on the displacement information of the analysis target to be adjusted to obtain an adjusted analysis target, if j<J, setting j=j+1, and returning to step 52, otherwise proceeding to step 56; Step 55, delete the jth first target to be analyzed, if j<J, set j=j+1, return to the step 52, otherwise go to step 56; Step 56: Generate an adjusted video frame based on all adjusted analysis targets and the third target to be analyzed, determine whether the adjustment of all the second video frames is completed, and if so, output the adjusted video segment; if not, define the adjusted video frame as the reference frame, and return to step 51, wherein the third target to be analyzed is the second target to be analyzed that has not been deleted from the analysis target to be adjusted.

Images