Multi-modal target data intelligent processing system

Abstract

The invention relates to the technical field of data processing, in particular to a multi-modal target data intelligent processing system, which comprises a target data acquisition module used for acquiring target field information and associated personnel information; the video data extraction module is used for extracting a video target area, analyzing video frame change characteristics and extracting an on-site video image; the image information analysis module is used for analyzing the image local features and the image overall features, and is also used for analyzing the obvious degree of the local features; the personnel information analysis module is used for storing associated personnel information, counting text matching times and the number of matched personnel and analyzing personnel associated parameters; and the data storage management module is used for analyzing the feature association degree and storing the target site information and the associated personnel information according to the feature association degree. According to the invention, intelligent storage processing of the target data is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a multi-modal target data intelligent processing system. Background Art

[0002] Target data processing mainly relies on the development of big data technology. With the explosive growth of data volume, reasonable data classification, storage and processing have become particularly important. By using big data technology to collect, store, process and analyze target data, massive data can be reasonably divided to improve data processing efficiency and quality.

[0003] Chinese Patent Publication No.: CN117763174A discloses a multimodal retrieval method, device and storage medium, including: determining image information and text information as retrieval input information; encoding the text information using a pre-set first encoding module to determine a first text feature belonging to a text feature space corresponding to the text information; migrating the first text feature using a pre-set first migration module to determine a corresponding second text feature belonging to an image feature space, and merging the first text feature and the second text feature to generate a third text feature; encoding the image information to generate an image feature corresponding to the image information; merging the third text feature with the image feature to generate a combined image-text feature; and searching based on the combined image-text feature to obtain a search result corresponding to the image information and the text information. This invention realizes the classification analysis of content features in image data, but does not realize the classification storage of target data by integrating video data, image data and text data, and has the problems of low efficiency in analyzing target data and inaccurate storage and processing of target data. Summary of the invention

[0004] The object of the present invention is to provide a multi-modal target data intelligent processing system to solve at least one of the problems existing in the prior art.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A multi-modal target data intelligent processing system, comprising: Target data collection module, used to collect target site information and related personnel information; A video data extraction module is used to extract the video target area according to the target scene information, and is also used to analyze the video frame change characteristics according to the target scene information, and extract the scene video image according to the video frame change characteristics and the video target area; An image information analysis module is used to analyze the local features and overall features of the image according to the target scene information, and is also used to analyze the degree of local feature prominence according to the local features of the image; A personnel information analysis module is used to store the associated personnel information and match the associated personnel information with the preset extracted keywords to count the number of text matches and the number of matching personnel, and also to analyze the personnel association parameters according to the number of text matches and the number of matching personnel; The data storage management module is used to analyze the feature correlation according to the local features of the image, the overall features of the image, the on-site video image and the personnel-related parameters, and store the target on-site information and the related personnel information according to the feature correlation.

[0006] Further, the video data extraction module is provided with a change feature analysis unit, which is used to extract an image frame every K frames in the on-site monitoring video as a video frame image, and number the video frame images according to the extraction order to obtain a video frame number, wherein K represents a video extraction parameter; The change feature analysis unit analyzes the video frame change feature according to the video frame image to obtain the video frame change feature Q(k), where k represents the video frame number.

[0007] Furthermore, the video data extraction module is also provided with a video image extraction unit, which is used to extract the on-site video image according to the video frame change feature Q(k). If there is a target area in the on-site monitoring video and Q(k)≤q, the video image extraction unit currently analyzes the video frame image corresponding to the video frame change feature as the on-site video image; otherwise, the video image extraction unit does not extract the on-site video image; wherein q represents the change feature threshold.

[0008] Furthermore, the image information analysis module is provided with a division region analysis unit, which is used to analyze the local features of the image according to the image division regions to obtain the local features of the image W(v), where v represents the division number; The divided region analysis unit analyzes the local feature obviousness according to the local feature W(v) of the image, and the local feature obviousness includes the local feature not obvious and the local feature obvious.

[0009] Furthermore, the image information analysis module is also provided with a picture feature analysis unit, which is used to analyze the overall image features according to the on-site grayscale image to obtain the overall image features A(i), wherein i represents the on-site grayscale image number.

[0010] Furthermore, the personnel information analysis module matches the target text with preset extraction keywords, and counts the number of occurrences of the text extraction keywords in the target text as the text matching number N1(j). If N1(j)>0, the personnel information analysis module sets the currently analyzed associated personnel information as the matched personnel; if N1(j)=0, the personnel information analysis module sets the currently analyzed associated personnel information as the unmatched personnel; the personnel information analysis module counts the number of associated personnel information of the matched personnel as the matched personnel number NR2, and analyzes the personnel association parameters according to the text matching number N1(j) and the matched personnel number NR2 to obtain the personnel association parameters R(j), where j represents the personnel number.

[0011] Furthermore, the data storage management module is provided with a feature association analysis unit, which is used to analyze the feature association degree according to the overall image feature A(i), the on-site video image and the personnel association parameter R(j) to obtain the feature association degree F(m).

[0012] Furthermore, the data storage management module is also provided with a local feature matching unit, which is used to take the split number corresponding to the image split area with obvious local features as the obvious feature number set U1, and analyze the local feature correlation according to the obvious feature number set U1, the local feature W(v) of the image and the overall feature A(i) of the image, wherein the local feature correlation includes weak local feature correlation and strong local feature correlation, and when the local feature correlation is strong, the feature correlation degree analysis process is processed, and the feature correlation degree after processing is F1(m).

[0013] Furthermore, the data storage management module is further provided with a storage matching analysis unit, which is used to count the number of stored target scene information and associated personnel information that satisfy |F(m) / F(z)-1|≤α1 as the storage matching number NF, and extract the stored target scene information that satisfies |A(i) / A(z)-1|≤α1 and has the same photo type as the matching scene information, and the storage matching analysis unit counts the number of matching scene information of the first storage type as the first matching number NA1, and counts the number of matching scene information of the second storage type as the second matching number NA2, wherein F(z) represents the feature correlation degree corresponding to the stored target scene information and the associated personnel information, z represents the storage information number, A(z) represents the overall image feature corresponding to the stored target scene information, and α1 represents the first storage matching threshold; The storage matching analysis unit analyzes the storage association according to the storage matching number NF, the first matching number NA1 and the second matching number NA2, wherein the storage association includes strong storage association and weak storage association, and when the storage association is strong, processes the analysis process of the local feature matching parameters, and the processed local feature matching parameters are B1.

[0014] Furthermore, the data storage management module is also provided with a data storage management unit, which is used to analyze the storage type of the target site information and the associated personnel information according to the feature correlation degree, and the storage type includes one category and two categories.

[0015] The beneficial effects of the present invention are as follows: through the target data acquisition module to collect the target site information and related personnel information, and the other modules to analyze the collected data, the storage of the collected data is managed by integrating video data, image data and text data, and the analysis of the characteristics of each data and the correlation between the data is realized, and the accuracy of the correlation analysis between the data is improved, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the storage and processing of the target data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 Schematic diagram of the structure of the multimodal target data intelligent processing system of this embodiment.

[0018] Figure 2 Schematic diagram of the structure of the video data extraction module in this embodiment.

[0019] Figure 3 Schematic diagram of the structure of the image information analysis module of this embodiment.

[0020] Figure 4 This is a structural diagram of the data storage management module of this embodiment. DETAILED DESCRIPTION

[0021] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0022] It should be noted that, although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0023] See also Figure 1 As shown, it is a multi-modal target data intelligent processing system of this embodiment, including: The target data acquisition module is used to collect target scene information and related personnel information. The target scene information includes the scene location, scene photo information and scene monitoring video. The scene photo information includes photo type and scene grayscale image. The photo type is data about the content of the scene photo. The scene grayscale image is the grayscale image of the scene photo. The scene monitoring video is the monitoring video data of the target scene. The related personnel information includes personnel identity information and target text. The personnel identity information includes but is not limited to name, gender, age, ID number and other data. The target scene information and related personnel information are collected by user interactive input.

[0024] Specifically, the target scene information in this embodiment is the scene data of a target location analyzed by the system, and the associated personnel information is the personnel information that may be associated with the target scene.

[0025] Specifically, this embodiment is applied to a cloud server storing target data, and by analyzing the correlation between target data and data characteristics, classified storage of target data is achieved to ensure efficiency in subsequent retrieval and analysis of target data.

[0026] It should be noted that the target information (including but not limited to target device information, target personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the target or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant national and regional laws, regulations and standards.

[0027] Please continue reading Figure 1 As shown, the multimodal target data intelligent processing system also includes: The video data extraction module is used to extract the video target area according to the target scene information, and is also used to analyze the video frame change characteristics according to the target scene information, and extract the scene video image according to the video frame change characteristics and the video target area. The video data extraction module is connected to the target data acquisition module.

[0028] Specifically, in this embodiment, when analyzing image data, the pixel point in the lower left corner of the image is used as the coordinate origin, and the two sides adjacent to the coordinate origin are used as the x-axis and y-axis respectively. The x-axis increases from left to right, and the y-axis increases from bottom to top. The unit length is 1 pixel. A plane rectangular coordinate system is established, and the coordinate points are used to represent the position of each pixel in the image.

[0029] See also Figure 2As shown, the video data extraction module includes: The target area extraction unit is used to extract the video target area according to the on-site monitoring video, and the video target area includes a human body area and a target area.

[0030] Specifically, the target region extraction unit in this embodiment uses a target detection model to extract the video target region, and has extracted the video target region in the on-site monitoring video.

[0031] Specifically, the target detection model described in this embodiment is a target detection model based on YoloV8. The target detection model is trained using a human detection dataset and a vehicle detection dataset to achieve the extraction of human areas and vehicle areas in on-site surveillance videos. In this embodiment, there is no specific limitation on the setting of the detection dataset, such as the use of VOC datasets and COCO datasets.

[0032] Specifically, in this embodiment, the target area extraction unit analyzes the on-site surveillance video to extract the target area in the on-site surveillance video, thereby realizing the extraction of people and vehicles in the on-site surveillance video, and ensuring the accuracy of target extraction in the surveillance video data, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the target data storage and processing.

[0033] Please continue reading Figure 2 As shown, the video data extraction module also includes: The change feature analysis unit is used to analyze the change features of the video frame according to the on-site monitoring video, and the change feature analysis unit is connected to the target area extraction unit.

[0034] Specifically, the change feature analysis unit in this embodiment extracts one frame of image every K frames from the on-site monitoring video as a video frame image, and numbers the video frame images in the extraction order to obtain a video frame number, where K represents a video extraction parameter, 20≤K≤30. It can be understood that the value of the video extraction parameter is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as the extraction of the video frame image is satisfied. The optimal value of the video extraction parameter is: K=27.

[0035] Specifically, the change feature analysis unit described in this embodiment analyzes the video frame change feature according to the video frame image, sets the video frame change feature to Q(k), and sets Q(k)=σ1(k) / [σ1(k-1)+1], wherein σ1(k) represents the image feature of the video frame image, and k represents the video frame number.

[0036] Specifically, the image feature of the video frame image in this embodiment is the standard deviation of the grayscale value of the pixel in the video frame image, which is calculated as follows: , L k (x, y) represents the gray value of each pixel in the video frame image, NXY represents the number of pixels in the video frame image, Avg(L k (x, y)) represents the average grayscale value of the pixels in the video frame image. It is understandable that the setting of the image features of the video frame image is not specifically limited in this embodiment, and those skilled in the art can freely set it, such as setting the image features of the video frame image to image features calculated by image feature extraction algorithms such as SIFT and HOG.

[0037] Specifically, in this embodiment, the change feature analysis unit extracts the video frame image to realize the extraction of part of the data used for analysis in the monitoring video, thereby analyzing the video frame change features, and using the video frame change features to represent the differences in image features between different video frames, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the target data storage and processing.

[0038] Please continue reading Figure 2 As shown, the video data extraction module also includes: The video image extraction unit is used to extract the on-site video image according to the video frame change characteristics and the video target area, and the video image extraction unit is connected to the change characteristic analysis unit.

[0039] Specifically, the video image extraction unit in this embodiment extracts the on-site video image according to the video frame change feature. If there is a target area in the on-site monitoring video and Q(k)≤q, the video image extraction unit currently analyzes the video frame image corresponding to the video frame change feature as the on-site video image; otherwise, the video image extraction unit does not extract the on-site video image; wherein q represents the change feature threshold, 0.5≤q≤0.8. It can be understood that the value of the change feature threshold is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as it satisfies the extraction of the on-site video image. The optimal value of the change feature threshold is: q=0.6.

[0040] Specifically, in this embodiment, the video image extraction unit analyzes the video change characteristics and the video target area to extract images in the on-site surveillance video where the detection target exists and the extracted video frame image change differences are large as on-site video images, thereby ensuring the diversity of the extracted on-site video images, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the target data storage and processing.

[0041] Please continue reading Figure 1 As shown, the multimodal target data intelligent processing system also includes: The image information analysis module is used to analyze the local features and the overall features of the image according to the target scene information. The image information analysis module is connected to the video data extraction module.

[0042] See also Figure 3 As shown, the image information analysis module includes: The scene image splitting unit is used to split the scene grayscale image to obtain image splitting areas.

[0043] Specifically, the scene image splitting unit in this embodiment creates a rectangular area with a pixel size of u×u, and overlaps the lower left corner of the rectangular area with the lower left corner pixel in the scene grayscale image, extracts the corresponding pixel in the rectangular area as an image splitting area, and translates the rectangular area to the right by one pixel after each extraction. If the rectangular area translates to the right to the right boundary position of the scene grayscale image, the rectangular area is translated upward by one pixel. If the rectangular area translates to the left to the left boundary position of the scene grayscale image, the rectangular area is translated upward by one pixel until the rectangular area translates to the left to the upper left boundary position of the scene grayscale image or translates to the right to the upper right boundary position of the scene grayscale image, so as to extract (NX-u)×(NY-u) image splitting areas, and number them according to the extraction order of the image splitting areas to obtain the splitting number, wherein u represents the splitting parameter, 4≤u≤10, NX represents the number of pixels in the x-axis direction of the scene grayscale image, and NY represents the number of pixels in the y-axis direction of the scene grayscale image. The optimal value of the splitting parameter is u=5.

[0044] Specifically, in this embodiment, the scene grayscale image is split by the scene image splitting unit to obtain multiple image splitting areas, thereby increasing the diversity of the system's analysis of the scene grayscale image and improving the accuracy of the analysis of each area in the scene grayscale image, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the target data storage and processing.

[0045] Please continue reading Figure 3 As shown, the image information analysis module also includes: The divided area analysis unit is used to analyze the local features of the image according to the image split areas, and to analyze the degree of local feature prominence according to the local features of the image. The divided area analysis unit is connected to the on-site image splitting unit.

[0046] Specifically, the divided area analysis unit described in this embodiment analyzes the local features of the image according to the image split area, sets the local features of the image to W(v), and sets W(v)=σ2(v), wherein σ2(v) represents the image features of the image split area, and v represents the split number.

[0047] Specifically, the image features of the image split area described in this embodiment are analyzed in the same manner as the image features of the video frame image, and will not be described in detail in this embodiment.

[0048] Specifically, the region division analysis unit in this embodiment analyzes the local feature prominence according to the local features of the image. If W(v) / Avg(L i (x, y))≤w, the partition region analysis unit determines that the local features of the split region of the current analysis image are not obvious; otherwise, the partition region analysis unit determines that the local features of the split region of the current analysis image are obvious; wherein Avg(L i (x, y)) represents the average grayscale value of the pixel in the grayscale image of the scene, w represents the threshold of the degree of prominence, 0.05≤w≤0.15. It can be understood that the value of the threshold of the degree of prominence is not specifically limited in this embodiment, and can be freely set by those skilled in the art, as long as it satisfies the analysis of the degree of prominence of the local features. The optimal value of the threshold of the degree of prominence is: w=0.1.

[0049] Specifically, in this embodiment, the image segmentation area is analyzed by the divided area analysis unit to analyze the local features of the image, and the image features of each image segmentation area are represented by the local features of the image, so as to analyze the degree of local feature significance and realize the extraction of image segmentation areas with obvious local features, thereby improving the system's analysis efficiency of target data and improving the accuracy of storage and processing of target data.

[0050] Please continue reading Figure 3 As shown, the image information analysis module also includes: The picture feature analysis unit is used to analyze the overall features of the image according to the on-site photo information, and the picture feature analysis is connected to the divided area analysis unit.

[0051] Specifically, the picture feature analysis unit in this embodiment analyzes the overall image feature according to the on-site grayscale image, sets the overall image feature to A(i), and sets A(i)=σ3(i), where σ3(i) represents the image feature of the on-site grayscale image, and i represents the on-site grayscale image number. The on-site grayscale image number is defined as a number for distinguishing different on-site photo information.

[0052] Specifically, the image features of the on-site grayscale image described in this embodiment are analyzed in the same manner as the image features of the video frame image, and will not be specifically described in this embodiment.

[0053] Please continue reading Figure 1 As shown, the multimodal target data intelligent processing system also includes: The personnel information analysis module is used to store the associated personnel information and match the associated personnel information with the preset extracted keywords to count the number of text matches and the number of matching personnel, and is also used to analyze the personnel association parameters according to the number of text matches and the number of matching personnel. The personnel information analysis module is connected to the target data acquisition module.

[0054] Specifically, the personnel information analysis module in this embodiment matches the target text with the preset extraction keywords, and counts the number of times the text extraction keywords appear in the target text as the number of text matching. If N1(j)>0, the personnel information analysis module sets the current analysis-related personnel information as the matching personnel; if N1(j)=0, the personnel information analysis module sets the current analysis-related personnel information as the unmatched personnel; the personnel information analysis module counts the number of related personnel information of the matching personnel as the number of matching personnel, and analyzes the personnel association parameter according to the number of text matching and the number of matching personnel, and sets the personnel association parameter to R(j), setting R(j)=N1(j)×lg[NR1 / (NR2+1)] / N2(j), wherein N1(j) represents the number of text matching, N2(j) represents the number of words in the target text, NR1 represents the number of related personnel information, NR2 represents the number of matching personnel, and j represents the personnel number. The personnel number is defined as the number to distinguish different related personnel information.

[0055] Specifically, in this embodiment, the preset extraction keywords include but are not limited to the target person's name and on-site location, etc., which only need to meet the extraction of important content in the target text. For example, keywords such as time and license plate number can also be added.

[0056] Specifically, in this embodiment, the personnel information analysis module performs matching analysis on the associated personnel information and the preset extracted keywords to analyze the number of text matches and the number of matching personnel, thereby analyzing the personnel association parameters, and using the personnel association parameters to represent the association features between key contents in the target text, thereby realizing the analysis of the word frequency features in the target text, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the storage and processing of the target data.

[0057] Please continue reading Figure 1 As shown, the multimodal target data intelligent processing system also includes: The data storage management module is used to analyze the feature correlation according to the local features of the image, the overall features of the image, the on-site video image and the personnel-related parameters, and store the target on-site information and the related personnel information according to the feature correlation. The data storage association module is connected to the image information analysis module and the personnel information analysis module.

[0058] See also Figure 4As shown, the data storage management module includes: The feature association analysis unit is used to analyze the feature association degree according to the overall image characteristics, the on-site video image and the personnel association parameters.

[0059] Specifically, the feature correlation analysis unit in this embodiment analyzes the feature correlation degree according to the overall image features, the on-site video image and the personnel correlation parameters, sets the feature correlation degree to F(m), and sets , where H1(m) represents the vector of the overall image features, H1(m)=[A(i)], H2(m) represents the vector of the image features of the on-site video image, H2(m)=[σ1(k)], H3(m) represents the vector of personnel-related parameters, H3(m)=[R(j)], and m represents the acquisition data number. The acquisition data number is defined as the number that distinguishes the target on-site information and associated personnel information of different groups.

[0060] Specifically, in this embodiment, the feature association analysis unit analyzes the overall image features, on-site video images and personnel-related parameters to analyze the feature association degree, and uses the feature association degree to represent the association characteristics between the on-site monitoring video, on-site grayscale image and related personnel information, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the target data storage and processing.

[0061] Please continue reading Figure 4 As shown, the data storage management module also includes: The local feature matching unit is used to analyze the local feature relevance according to the local features of the image and the overall features of the image. The local feature matching unit is connected to the feature relevance analysis unit.

[0062] Specifically, the local feature matching unit in this embodiment uses the split number corresponding to the image split area with obvious local features as the obvious feature number set, and analyzes the local feature correlation according to the obvious feature number set, the local features of the image and the overall features of the image. If B≤b, the local feature matching unit determines that the local feature correlation is weak; otherwise, the local feature matching unit determines that the local feature correlation is strong, and processes the feature correlation analysis process. The processed feature correlation is F1(m), and F1(m)=F(m)×e is set. 1-B ; Where B represents the local feature matching parameter, set , U1 represents the obvious feature number set, NU1 represents the number of split numbers in the obvious feature number set, b represents the local correlation threshold, 0.7≤b≤0.9. It can be understood that the value of the local correlation threshold is not specifically limited in this embodiment, and those skilled in the art can freely set it as long as it satisfies the analysis of the local feature correlation. The optimal value of the local correlation threshold is: b=0.8.

[0063] Specifically, in this embodiment, the local feature matching unit analyzes the local features and the overall features of the image to analyze the local feature matching parameters, and uses the local feature matching parameters to represent the difference relationship between the overall features of the scene grayscale image and the local features of the image in the image segmentation area where the local features are obvious, so as to analyze the local feature correlation. When the local feature correlation is strong, the feature correlation analysis process is processed so that the processed feature correlation is related to the local features of the image, thereby increasing the accuracy of image data processing, thereby improving the system's analysis efficiency of the target data and improving the accuracy of target data storage and processing.

[0064] Please continue reading Figure 4 As shown, the data storage management module also includes: The storage matching analysis unit is used to match the stored target scene information and related personnel information according to the feature association degree, and analyze the storage association according to the matching result. The storage matching analysis unit is connected to the local feature matching unit.

[0065] Specifically, the storage matching analysis unit in this embodiment counts the number of stored target scene information and associated personnel information that satisfy |F(m) / F(z)-1|≤α1 as the storage matching number, and extracts the stored target scene information that satisfies |A(i) / A(z)-1|≤α1 and has the same photo type as the matching scene information. The storage matching analysis unit counts the number of matching scene information with storage type of one as the first matching number, and counts the number of matching scene information with storage type of two as the second matching number, wherein F(z) represents the feature correlation degree corresponding to the stored target scene information and the associated personnel information, z represents the storage information number, A(z) represents the overall image feature corresponding to the stored target scene information, and α1 represents the first storage matching threshold, 0.05≤α1≤0.1. It can be understood that the value of the first storage matching threshold is not specifically limited in this embodiment, and can be freely set by those skilled in the art. The optimal value of the first storage matching threshold is: α1=0.08.

[0066] Specifically, the storage matching analysis unit in this embodiment analyzes the storage association according to the storage matching number, the first matching number and the second matching number. If [NA2 / (NA1+NA2)] NF / Nm>α2, the storage matching analysis unit determines that the storage association is weak; otherwise, the storage matching analysis unit determines that the storage association is strong, and processes the analysis process of the local feature matching parameters. The processed local feature matching parameters are B1, and B1=B×(NA1+NA2) / NA2 is set; wherein NA1 represents the first matching number, NA2 represents the second matching number, NF represents the storage matching number, Nm represents the number of stored target site information and associated personnel information, and α2 represents the second storage matching threshold, 1≤α2≤1.2. It can be understood that in this embodiment, the value of the second storage matching threshold is not specifically limited, and those skilled in the art can freely set it, as long as the analysis of the storage association is satisfied. The optimal value of the second storage matching threshold is α2=1.1.

[0067] Specifically, in this embodiment, the storage matching analysis unit analyzes the stored target scene information and related personnel information to analyze the storage matching number, the first matching number and the second matching number, and realizes the extraction and analysis of similar data of the currently analyzed collected data, thereby analyzing the storage correlation. When the storage correlation is strong, the analysis process of the local feature matching parameters is processed to make the local feature matching parameters related to the correlation between the current collected data and the stored data, thereby improving the system's analysis efficiency of the target data and improving the accuracy of the target data storage and processing.

[0068] Please continue reading Figure 4 As shown, the data storage management module also includes: The data storage management unit is used to analyze the storage type of the target scene information and the associated personnel information according to the feature association degree, and the data storage management unit is connected to the storage matching analysis unit.

[0069] Specifically, the data storage management unit in this embodiment analyzes the storage type of the target scene information and the associated personnel information according to the feature correlation degree. If F(m)≤f, the data storage management unit determines that the storage type of the target scene information and the associated personnel information is one type; otherwise, the data storage management unit determines that the storage type of the target scene information and the associated personnel information is two types; wherein f represents the storage management threshold, 0.6≤f≤0.9. It can be understood that in this embodiment, the value of the storage management threshold is not specifically limited, and those skilled in the art can freely set it, as long as it meets the judgment of the storage type. The optimal value of the storage management threshold is: f=0.8.

[0070] Specifically, in this embodiment, the storage type is one, which means that the collected target site information and related personnel information are cold data, and the currently collected data can be stored in an ordinary storage device. The storage type is two, which means that the collected target site information and related personnel information are hot data, which are key data for data analysis. The currently collected data can be stored in a high-performance storage device to improve the efficiency of subsequent data retrieval and analysis.

[0071] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the embodiments here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A multi-modal target data intelligent processing system, characterized in that: include: Target data collection module, used to collect target site information and related personnel information; A video data extraction module is used to extract the video target area according to the target scene information, and is also used to analyze the video frame change characteristics according to the target scene information, and extract the scene video image according to the video frame change characteristics and the video target area; An image information analysis module is used to analyze the local features and overall features of the image according to the target scene information, and is also used to analyze the degree of local feature prominence according to the local features of the image; A personnel information analysis module is used to store the associated personnel information and match the associated personnel information with the preset extracted keywords to count the number of text matches and the number of matching personnel, and also to analyze the personnel association parameters according to the number of text matches and the number of matching personnel; The data storage management module is used to analyze the feature correlation according to the local features of the image, the overall features of the image, the on-site video image and the personnel-related parameters, and store the target on-site information and the related personnel information according to the feature correlation.

2. The multimodal target data intelligent processing system according to claim 1, characterized in that: The video data extraction module is provided with a change feature analysis unit, which is used to extract an image frame every K frames from the on-site monitoring video as a video frame image, and number the video frame images according to the extraction order to obtain a video frame number, wherein K represents a video extraction parameter; The change feature analysis unit analyzes the video frame change feature according to the video frame image to obtain the video frame change feature Q(k), where k represents the video frame number.

3. The multimodal target data intelligent processing system according to claim 2, characterized in that: The video data extraction module is also provided with a video image extraction unit, which is used to extract the on-site video image according to the video frame change feature Q(k). If there is a target area in the on-site monitoring video and Q(k)≤q, the video image extraction unit currently analyzes the video frame image corresponding to the video frame change feature as the on-site video image; otherwise, the video image extraction unit does not extract the on-site video image; wherein q represents the change feature threshold.

4. The multimodal target data intelligent processing system according to claim 3, characterized in that: The image information analysis module is provided with a division region analysis unit, which is used to analyze the local features of the image according to the image division regions to obtain the local features of the image W(v), where v represents the division number; The divided region analysis unit analyzes the local feature obviousness according to the local feature W(v) of the image, and the local feature obviousness includes the local feature not obvious and the local feature obvious.

5. The multimodal target data intelligent processing system according to claim 4, characterized in that: The image information analysis module is also provided with a picture feature analysis unit, which is used to analyze the overall image features according to the on-site grayscale image to obtain the overall image features A(i), wherein i represents the on-site grayscale image number.

6. The multi-modal target data intelligent processing system according to claim 5, characterized in that: The personnel information analysis module matches the target text with the preset extraction keywords, and counts the number of occurrences of the text extraction keywords in the target text as the text matching number N1(j). If N1(j)>0, the personnel information analysis module sets the currently analyzed associated personnel information as the matching personnel; if N1(j)=0, the personnel information analysis module sets the currently analyzed associated personnel information as the unmatched personnel; the personnel information analysis module counts the number of associated personnel information of the matching personnel as the matching personnel number NR2, and analyzes the personnel association parameters according to the text matching number N1(j) and the matching personnel number NR2 to obtain the personnel association parameters R(j), where j represents the personnel number.

7. The multimodal target data intelligent processing system according to claim 6, characterized in that: The data storage management module is provided with a feature association analysis unit, which is used to analyze the feature association degree according to the overall image feature A(i), the on-site video image and the personnel association parameter R(j) to obtain the feature association degree F(m).

8. The multimodal target data intelligent processing system according to claim 7, characterized in that: The data storage management module is also provided with a local feature matching unit, which is used to take the split number corresponding to the image split area with obvious local features as the obvious feature number set U1, and analyze the local feature correlation according to the obvious feature number set U1, the local feature W(v) of the image and the overall feature A(i) of the image. The local feature correlation includes weak local feature correlation and strong local feature correlation. When the local feature correlation is strong, the feature correlation degree analysis process is processed, and the feature correlation degree after processing is F1(m).

9. The multimodal target data intelligent processing system according to claim 8, characterized in that: The data storage management module is also provided with a data storage management unit, which is used to analyze the storage type of the target scene information and the associated personnel information according to the feature correlation degree, and the storage type includes one category and two categories.

10. The multi-modal target data intelligent processing system according to claim 9, characterized in that: The data storage management module is further provided with a storage matching analysis unit, which is used to count the number of stored target scene information and associated personnel information that satisfy |F(m) / F(z)-1|≤α1 as the storage matching number NF, and extract the stored target scene information that satisfies |A(i) / A(z)-1|≤α1 and has the same photo type as the matching scene information, and the storage matching analysis unit counts the number of matching scene information of the first storage type as the first matching number NA1, and counts the number of matching scene information of the second storage type as the second matching number NA2, wherein F(z) represents the feature correlation degree corresponding to the stored target scene information and the associated personnel information, z represents the storage information number, A(z) represents the overall image feature corresponding to the stored target scene information, and α1 represents the first storage matching threshold; The storage matching analysis unit analyzes the storage association according to the storage matching number NF, the first matching number NA1 and the second matching number NA2, wherein the storage association includes strong storage association and weak storage association, and when the storage association is strong, processes the analysis process of the local feature matching parameters, and the processed local feature matching parameters are B1.

Images