A city-level large-scale intelligent video analytics method and cluster architecture

Abstract

This invention provides a city-level large-scale intelligent video analysis method and cluster architecture. The method includes a video information preprocessing process, a request analysis process, a resource scheduling process, and a video analysis process. The cluster architecture includes a video information acquisition terminal, a user terminal, and an intelligent video information analysis platform. The intelligent video information analysis platform includes a video information preprocessing system, a request analysis system, an algorithm resource scheduling system, an algorithm module, an intelligent video analysis engine, a unified data sharing and exchange system, and a business application microservice system. The video information preprocessing system includes a video source tag management module and a tag stream management module. The request analysis system includes a request receiving module, a content request recognition module, a time-sharing invocation module, and an algorithm request recognition module. This invention analyzes massive amounts of existing video information to quickly and accurately generate and push the required analysis results to different users.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence image processing, and more specifically, to a city-level large-scale intelligent video analysis method and cluster architecture. Background Technology

[0002] The video surveillance market has experienced explosive growth over the past decade. While surveillance brings us a sense of security, it has also raised issues such as the storage and analysis of massive amounts of data, as well as the problem of valuable information being buried and difficult to extract.

[0003] Currently, smart cameras for face capture and license plate capture are being widely used in industries such as transportation. However, these smart cameras are expensive and have limited functionality, making it difficult to completely replace existing cameras.

[0004] In addition, existing intelligent video analytics platforms have limited concurrent processing capabilities and relatively simple business functions, making it difficult to meet the needs. There is a lack of a city-level large-scale intelligent video cluster analytics platform to provide effective video discovery, analysis, and decision-making application services for city-level management issues. This leads to redundant construction of monitoring front-ends and small-scale analysis platforms by various departments in urban management, which not only wastes resources but also creates information silos, resulting in a large amount of video being idle and unable to effectively play its monitoring value.

[0005] Currently, with the continuous advancement of artificial intelligence technology, some solutions have been proposed for large-scale intelligent video analysis. For example, CN113259451A describes a cluster processing architecture and method for intelligent analysis of large-scale monitoring nodes. The cluster processing architecture includes multiple analysis devices, which essentially distributes large-scale video streams to multiple analysis devices for processing.

[0006] However, large-scale intelligent video analytics at the city level is not simply a matter of piling up intelligent analysis servers. It is characterized by a large amount of video information, diverse and complex user requests, and a constant emergence of new user requests. How to analyze massive amounts of existing video information and quickly and accurately generate and push the required analysis results to different users has become an urgent problem to be solved.

[0007] In order to solve the above problems, people have been seeking an ideal technological solution. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by providing a city-level large-scale intelligent video analysis method and cluster architecture.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] The first aspect of this invention provides a city-level large-scale intelligent video analysis method, the method comprising the following steps;

[0011] Step 1: Obtain the video source tags and new tags of the original video information, and generate and store a structured tag stream based on the video source tags and the new tags;

[0012] The original video information comes from the corresponding video information acquisition terminal, the video source tag includes the video acquisition time and the video information acquisition terminal position, and the new tag includes a scene algorithm identifier, which is used to identify the algorithm information pre-configured for the original video information.

[0013] Step 2: Determine if a video task analysis request has been received from the user's client.

[0014] If so, then parse out the user identity tag, content request tag, time tag and algorithm tag in the video task analysis request, and proceed to step 3;

[0015] The content request tag includes the endpoint of the video information to be analyzed, the time tag includes the analysis frequency and the video acquisition time to be analyzed, and the algorithm tag is used to identify the algorithm type corresponding to the video task analysis request.

[0016] Step 3: Read the structured tag stream, extract the video information acquisition endpoints from the structured tag stream, and perform point matching between the extracted video information acquisition endpoints and the video information acquisition endpoints to be analyzed in the content request tag;

[0017] Based on the location matching results, the original video information is initially filtered to select the target video information;

[0018] Step 4: Extract the corresponding video segments from the target video information based on the analysis frequency in the time tags;

[0019] The extracted video segments are decoded to obtain the image stream to be analyzed;

[0020] Step 5: Match the video capture time of the image stream to be analyzed based on the video capture time in the time tag, and filter out the target image stream from the image stream to be analyzed;

[0021] Step 6: Obtain the target scene algorithm identifier, and determine whether the target scene algorithm identifier matches the algorithm tag of the video task analysis request. If they match, generate an algorithm scheduling task sheet.

[0022] Wherein, the target scene algorithm identifier refers to the scene algorithm identifier associated with the target image stream;

[0023] Step 7: According to the algorithm scheduling task list, extract the target algorithm module from the preset basic algorithm modules; wherein, the basic algorithm modules are set in a one-to-one correspondence with the algorithm tags;

[0024] The target algorithm module is invoked to analyze and process the target image stream, generating image frame analysis results;

[0025] The image frame analysis results include user identity tags and event analysis results. The event analysis results include event images, event occurrence time, event occurrence location, event type, event confidence level, and target area coordinates.

[0026] Step 8: Construct an index code for the image frame analysis results and store it in association with the image frame analysis results;

[0027] The index code is set to correspond to the user identity tag in the video task analysis request;

[0028] Step 9: Based on the user identity tag corresponding to the index code, push the image frame analysis result to the corresponding user terminal. Different user identity tags correspond to different user terminals.

[0029] A second aspect of the present invention provides a city-level large-scale intelligent video analysis cluster architecture, which includes a video information acquisition terminal, a user terminal, and an intelligent video information analysis platform. The intelligent video information analysis platform includes a video information preprocessing system, a request analysis system, an algorithm resource scheduling system, an algorithm module, an intelligent video analysis engine, a unified data sharing and exchange system, and a business application microservice system. The video information preprocessing system includes a video source tag management module and a tag stream management module. The request analysis system includes a request receiving module, a content request recognition module, a time-sharing invocation module, and an algorithm request recognition module.

[0030] The video information acquisition terminal is communicatively connected to the video information preprocessing system, and is used to acquire video information in real time and transmit the acquired raw video information to the video information preprocessing system.

[0031] The video source tag management module is used to obtain the video source tag of the original video information; wherein, the video source tag includes the video capture time and the video information capture endpoint.

[0032] The tag stream management module is used to acquire new tags for the original video information, generate a structured tag stream based on the video source tags and the new tags, and transmit the structured tag stream to the intelligent video analysis engine; wherein, the new tags include scene algorithm identifiers, which are used to identify algorithm information pre-configured for the original video information;

[0033] The user terminal is connected to the request analysis system and is used to generate different video task analysis requests and transmit them to the request analysis system.

[0034] The request receiving module is communicatively connected to both the user terminal and the intelligent video analysis engine. It is used to determine whether a video task analysis request has been received from the user terminal. If so, it parses out the user identity tag in the video task analysis request and transmits it to the intelligent video analysis engine. It is also used to transmit the received video task analysis request to the content request recognition module, the time-sharing calling module, and the algorithm request recognition module, respectively.

[0035] The content request recognition module is communicatively connected to both the request receiving module and the intelligent video analysis engine, and is used to parse the content request tag in the video task analysis request and transmit it to the intelligent video analysis engine; wherein, the content request tag includes the endpoint bit of the video information to be analyzed;

[0036] The time-sharing invocation module is communicatively connected to both the request receiving module and the intelligent video analysis engine, and is used to parse the time tags in the video task analysis request and transmit them to the intelligent video analysis engine; wherein, the time tags include the analysis frequency and the video acquisition time to be analyzed;

[0037] The algorithm request identification module is communicatively connected to both the request receiving module and the algorithm resource scheduling system. It is used to parse the algorithm tag in the video task analysis request and transmit it to the algorithm resource scheduling system. The algorithm tag is used to identify the algorithm type corresponding to the video task analysis request.

[0038] The intelligent video analysis engine is communicatively connected to the request analysis system, the video information preprocessing system, and the algorithm resource scheduling system. It is used to read the structured tag stream, extract video information acquisition endpoints from the structured tag stream, and perform point-to-point matching between the extracted video information acquisition endpoints and the video information acquisition endpoints to be analyzed in the content request tags. It is also used to perform preliminary filtering of the original video information based on the point-to-point matching results to select target video information; to extract corresponding video segments from the target video information based on the analysis frequency in the time tags; to decode the extracted video segments to obtain the image stream to be analyzed; and to perform video acquisition time matching of the image stream to be analyzed based on the video acquisition time in the time tags to select the target image stream.

[0039] The intelligent video analysis engine is also used to call the target algorithm module to analyze and process the target image stream and generate image frame analysis results; wherein, the image frame analysis results include user identity tags and event analysis results, and the event analysis results include event images, event occurrence time, event occurrence location, event type, event confidence level, and target area coordinates;

[0040] The algorithm resource scheduling system is communicatively connected to both the request analysis system and the intelligent video analysis engine. It is used to obtain a target scene algorithm identifier, determine whether the target scene algorithm identifier matches the algorithm tag of the video task analysis request, and if they match, generate an algorithm scheduling task list. It is also used to extract a target algorithm module from a preset set of basic algorithm modules based on the algorithm scheduling task list. The target scene algorithm identifier refers to a scene algorithm identifier associated with the target image stream, and the basic algorithm modules are set in a one-to-one correspondence with the algorithm tags.

[0041] The unified data sharing and exchange system is communicatively connected to the intelligent video analysis engine and is used to construct an index code for the image frame analysis results and store it in association with the image frame analysis results; wherein, the index code is set to correspond to the user identity tag in the video task analysis request;

[0042] The business application microservice system is connected to the unified data sharing and exchange system and the user terminal respectively. It is used to pre-store user identity tags and push the image frame analysis results to the corresponding user terminal according to the user identity tag corresponding to the index code.

[0043] This invention has outstanding substantive features and significant progress compared to the prior art, specifically:

[0044] 1) This invention analyzes massive amounts of existing video information through close coordination between various processes in the video information preprocessing process, request analysis process, resource scheduling process and video analysis process. It can quickly and accurately generate and push the required analysis results to different users, effectively solving the problem of difficulty in analyzing massive amounts of video information due to the large amount of video information, the diversity of user requests, the complexity of user requests and the continuous emergence of new user requests.

[0045] 2) This invention adopts a shared video and computing resources approach to maximize the value of surveillance videos. Through the construction of a large-scale city-level intelligent video analysis cluster platform, it shares the monitoring resources of various commissions and departments, adopts a microservice approach to meet the regulatory requirements of various commissions and departments, and upgrades the monitoring system intelligently by making the most of existing resources. Attached Figure Description

[0046] Figure 1 This is a time-series diagram of the city-level large-scale intelligent video analysis method of the present invention;

[0047] Figure 2 This is a flowchart illustrating the city-level large-scale intelligent video analysis method of the present invention;

[0048] Figure 3 This is a schematic diagram of the urban-level large-scale intelligent video analysis cluster architecture of the present invention;

[0049] Figure 4 This is a schematic diagram of the intelligent video analysis engine of the present invention;

[0050] Figure 5 This is a flowchart illustrating the method for dynamically expanding and shrinking the DCU hardware decoding resource usage of the present invention.

[0051] Figure 6 This is a schematic diagram of the emergency response method of the present invention. Detailed Implementation

[0052] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0053] Example 1

[0054] Appendix Figure 1 and 2 A schematic diagram of a city-level large-scale intelligent video analysis method is shown, which includes the following steps;

[0055] Step 1: Obtain the video source tags and new tags of the original video information, and generate a structured tag stream based on the video source tags and the new tags;

[0056] The original video information comes from the corresponding video information acquisition terminal. The video source tag includes video acquisition time, video information acquisition terminal location, video to which the unit belongs, IP address, username and password. The new tag includes scene algorithm identifier. The scene algorithm identifier is used to identify the algorithm information pre-configured for the original video information. The IP address, username and password are used to authenticate the accessed video information acquisition terminal.

[0057] Step 2: Determine if a video task analysis request has been received from the user's client.

[0058] If so, then parse out the user identity tag, content request tag, time tag and algorithm tag in the video task analysis request, and proceed to step 3;

[0059] Different user identity tags correspond to different user terminals;

[0060] The user identity tag includes the unit information, username and password; the content request tag includes the endpoint of the video information to be analyzed; the time tag includes the analysis frequency and the video collection time to be analyzed; and the algorithm tag is used to identify the algorithm type corresponding to the video task analysis request.

[0061] Step 3: Read the structured tag stream, extract the video information acquisition endpoints from the structured tag stream, and perform point matching between the extracted video information acquisition endpoints and the video information acquisition endpoints to be analyzed in the content request tag;

[0062] Based on the location matching results, the original video information is initially filtered to select the target video information;

[0063] Wherein, the video information acquisition endpoint bit in the video source tag corresponding to the target video information matches the video information acquisition endpoint bit to be analyzed in the content request tag;

[0064] Step 4: Extract the corresponding video segments from the target video information based on the analysis frequency in the time tags;

[0065] The extracted video segments are decoded to obtain the image stream to be analyzed;

[0066] Step 5: Match the video capture time of the image stream to be analyzed based on the video capture time in the time tag, and filter out the target image stream from the image stream to be analyzed;

[0067] The target image stream refers to the image stream to be analyzed whose video capture time in the video source tag matches the time tag of the video task analysis request;

[0068] Step 6: Obtain the target scene algorithm identifier, determine whether the target scene algorithm identifier matches the algorithm tag of the video task analysis request. If they match, generate an algorithm scheduling task sheet. If they do not match, continue to match the algorithm tags and end the algorithm tag matching when all algorithm tags do not match.

[0069] Wherein, the target scene algorithm identifier refers to the scene algorithm identifier associated with the target image stream;

[0070] Step 7: According to the algorithm scheduling task list, extract the target algorithm module from the preset basic algorithm modules; wherein, the basic algorithm modules are set in a one-to-one correspondence with the algorithm tags;

[0071] The target algorithm module is invoked to analyze and process the target image stream, generating image frame analysis results;

[0072] The image frame analysis results include user identity tags and event analysis results. The event analysis results include event images, event occurrence time, event occurrence location, event type, event confidence level, and target area coordinates.

[0073] Step 8: Construct an index code for the image frame analysis results and store it in association with the image frame analysis results;

[0074] The index code is set to correspond to the user identity tag in the video task analysis request;

[0075] Step 9: Based on the user identity tag corresponding to the index code, push the image frame analysis result to the corresponding user terminal. Different user identity tags correspond to different user terminals.

[0076] Furthermore, in step 1, when obtaining the video source tag and new tag of the original video information, the following is performed:

[0077] Step 101: Verify identity based on information such as the IP address of the video information acquisition terminal (e.g., camera) and the username of the video information acquisition terminal, and extract information such as the video acquisition time, video information acquisition terminal location, and video unit information after successful identity verification;

[0078] Step 102: Configure the scene algorithm identifier for the original video information based on the video acquisition time, video information acquisition endpoint location, and video unit information;

[0079] Step 103: Use the scene algorithm identifier configured in step 102 as a new tag for the original video information; specifically, the new tag is represented as new tag VIN'[scene algorithm identifier, etc.];

[0080] The video source tag of the original video information includes the video capture time, video information capture endpoint, video organization information, video information capture endpoint IP address, and the username and password corresponding to the video information capture endpoint. Specifically, the video source tag is represented as the video source tag VIN [video capture time, video information capture endpoint, video organization information, IP address, username and password, etc.].

[0081] The video information acquisition endpoint refers to the installation location of the video information acquisition terminal, corresponding to the location where the event occurred.

[0082] Specifically, the scene algorithm identifiers can be: motor vehicle illegal parking scene algorithm identifiers, non-motor vehicle illegal parking scene algorithm identifiers, storefront operation scene algorithm identifiers, road water accumulation scene algorithm identifiers, exposed loess scene algorithm identifiers, exposed garbage scene algorithm identifiers, crowd gathering scene algorithm identifiers, and people falling scene algorithm identifiers, etc.

[0083] To improve reliability, when accessing the raw video information collected by the video information acquisition terminal, the identity is manually verified and the corresponding scene algorithm identifier is configured.

[0084] Furthermore, in step 2, when obtaining the content request tag, time tag, and algorithm tag in the video task analysis request, the following is executed:

[0085] Step 201: The user inputs their analysis needs as a video task analysis request;

[0086] Step 202: Extract the data acquisition endpoints of the video information to be analyzed from the video task analysis request and generate content request tags;

[0087] Extract the analysis frequency and video acquisition time from the video task analysis request, and generate time tags;

[0088] Extract the video task analysis target from the video task analysis request and generate algorithm tags;

[0089] The video task analysis targets include analysis of illegal parking of motor vehicles, illegal parking of non-motor vehicles, outdoor business operations, road water accumulation, exposed loess, exposed garbage detection, crowd gathering detection, and people falling detection.

[0090] Furthermore, step 7 includes the following steps:

[0091] Step 701: Read the target image stream corresponding to the video task analysis request; wherein, the target image stream refers to a series of images that conform to the video task analysis request;

[0092] Step 702: Parse the algorithm tags in the algorithm scheduling task sheet, and perform algorithm type matching between the parsed algorithm tags and the algorithm tags corresponding to the basic algorithm modules;

[0093] Based on the algorithm type matching results, the target algorithm module is extracted from the preset basic algorithm modules;

[0094] The algorithm scheduling task sheet includes algorithm type information identified by an algorithm tag, and the algorithm type identified by the algorithm tag is consistent with the algorithm type corresponding to the target algorithm module.

[0095] Step 703: Based on the target algorithm module, perform algorithmic reasoning on the read target image stream to obtain image frame analysis results;

[0096] The image frame analysis results are JSON format string data, specifically including user identity tags, event images, event occurrence time, event occurrence location, event type, event confidence level, and target area coordinates;

[0097] Wherein, the user identity tag comes from the video task analysis request; the event occurrence time refers to the acquisition time corresponding to the event image; the event occurrence location refers to the spatial geographic coordinates, corresponding to the video information acquisition endpoint; the event type refers to the type of event that occurred in the event image; the event confidence level refers to the probability of the algorithm, such as the probability that a certain image is identified as event xx; and the target area coordinates refer to the pixel coordinates corresponding to the target area on the event image.

[0098] It should be noted that the city-level large-scale intelligent video analysis method in this embodiment includes a video information preprocessing process, a request analysis process, a resource scheduling process, and a video analysis process;

[0099] In the video information preprocessing process, video source tags and new tags of the original video information are generated. By storing the identification information of the original video information, the filtering efficiency and resource scheduling efficiency of the target image stream can be improved while avoiding the storage of massive image data.

[0100] During the request analysis process, the received video task analysis request is parsed to extract the user identity tag, content request tag, time tag and algorithm tag from the video task analysis request, thereby enhancing the correlation between the video task analysis request and the original video information;

[0101] During the resource scheduling process, the structured tag stream and the content request tag are matched at specific points. Based on the matching results, the original video information is initially screened to quickly and accurately filter out the target video information, thereby effectively reducing the amount of decoding computation.

[0102] Before decoding, corresponding video segments are extracted from the target video information based on the analysis frequency in the time tag, further reducing the decoding computation and thus improving the efficiency of large-scale intelligent video analysis.

[0103] After decoding, video capture time matching is performed based on the video capture time in the time tag, and the target image stream is filtered out from the image stream to be analyzed. Thus, by filtering again, the amount of computation required for target image stream analysis and processing is effectively reduced.

[0104] Based on the algorithm tags of the video task analysis request, algorithm type matching is performed. Based on the algorithm type matching results, the target algorithm module is extracted from the preset basic algorithm modules, thereby quickly finding the required basic algorithm module.

[0105] In the video analysis process, the selected target image stream is analyzed and processed based on the target algorithm module to generate image frame analysis results; an index code is constructed and stored in association with the image frame analysis results, thereby facilitating the subsequent push operation of image frame analysis results and improving the efficiency of analysis result push.

[0106] Therefore, this embodiment effectively solves the problem of difficulty in analyzing massive amounts of video information caused by the large amount of video information, diverse and complex user requests, and the continuous emergence of new user requests, through the close cooperation between the various processes in the video information preprocessing process, request analysis process, resource scheduling process, and video analysis process.

[0107] Example 2

[0108] It should be noted that users may have in-depth analysis needs. Therefore, based on Example 1, this example presents another method for large-scale intelligent video analysis at the city level.

[0109] Specifically, in step 9, before pushing the image frame analysis results to the corresponding user terminal, the following is also performed:

[0110] Based on the correlation between the image frame analysis results, a response result for the corresponding video task analysis request is generated and stored in association with the corresponding index code;

[0111] The response result refers to the in-depth analysis results of one or more video segments.

[0112] In one specific implementation, in addition to video task analysis targets such as illegal parking of motor vehicles, illegal parking of non-motor vehicles, operation outside shops, road water accumulation, exposed loess, exposed garbage detection, crowd gathering detection, people falling detection, face information detection, and license plate information detection, there may also be other video task analysis targets such as people trajectory analysis and vehicle trajectory analysis.

[0113] For other video task analysis targets such as personnel trajectory analysis and vehicle trajectory analysis, this embodiment can perform personnel trajectory analysis based on the facial information (or human body information) in the image frame analysis results, on the basis of various event analysis requests; or, perform vehicle trajectory analysis based on the license plate information, vehicle attribute information, etc. in the image frame analysis results.

[0114] Alternatively, based on information such as event type, event time, and event location in the image frame analysis results, statistical analysis can be performed on the distribution of various events, as well as the statistics and ranking of various high-incidence events.

[0115] Example 3

[0116] It should be noted that in practical applications, unexpected events may occur, and therefore users may have new analysis needs. Therefore, based on the above embodiments, this embodiment presents another method for large-scale intelligent video analysis at the city level, as shown in the appendix. Figure 6 As shown.

[0117] Specifically, the city-level large-scale intelligent video analysis method also performs:

[0118] The system detects in real time whether it receives configuration information containing sudden events. If so, it updates the preset basic algorithm module and adds a new algorithm module to realize the detection and identification of sudden events.

[0119] After adding new algorithm modules, new algorithm tags are set, and a mapping relationship is established between the new algorithm tags and the algorithm modules corresponding to the sudden events. When the video task analysis request includes the detection of sudden events, the city-level large-scale intelligent video analysis method can output the detection results of the sudden events in a timely and accurate manner, thereby enhancing the flexibility of intelligent video analysis.

[0120] Example 4

[0121] Based on Example 1, this example presents a city-level large-scale intelligent video analytics cluster architecture, as shown in the attached figure. Figure 3 As shown;

[0122] It includes a video information acquisition terminal, a user terminal, and an intelligent video information analysis platform. The intelligent video information analysis platform includes a video information preprocessing system, a request analysis system, an algorithm resource scheduling system, an algorithm module, an intelligent video analysis engine, a unified data sharing and exchange system, and a business application microservice system. The video information preprocessing system includes a video source tag management module and a tag stream management module. The request analysis system includes a request receiving module, a content request recognition module, a time-sharing invocation module, and an algorithm request recognition module.

[0123] The video information acquisition terminal is communicatively connected to the video information preprocessing system, and is used to acquire video information in real time and transmit the acquired raw video information to the video information preprocessing system.

[0124] The video source tag management module is used to obtain the video source tag of the original video information; wherein, the video source tag includes the video capture time and the video information capture endpoint.

[0125] The tag stream management module is used to acquire new tags for the original video information, generate a structured tag stream based on the video source tags and the new tags, and transmit the structured tag stream to the intelligent video analysis engine; wherein, the new tags include scene algorithm identifiers, which are used to identify algorithm information pre-configured for the original video information;

[0126] The user terminal is connected to the request analysis system and is used to generate different video task analysis requests and transmit them to the request analysis system; wherein, the video task analysis request includes user identity tags, content request tags, time tags and algorithm tags, and different user identity tags correspond to different user terminals;

[0127] The request receiving module is communicatively connected to both the user terminal and the intelligent video analysis engine. It is used to determine whether a video task analysis request has been received from the user terminal. If so, it parses out the user identity tag in the video task analysis request and transmits it to the intelligent video analysis engine. It is also used to transmit the received video task analysis request to the content request recognition module, the time-sharing calling module, and the algorithm request recognition module, respectively.

[0128] The content request recognition module is communicatively connected to both the request receiving module and the intelligent video analysis engine, and is used to parse the content request tag in the video task analysis request and transmit it to the intelligent video analysis engine; wherein, the content request tag includes the endpoint bit of the video information to be analyzed;

[0129] The time-sharing invocation module is communicatively connected to both the request receiving module and the intelligent video analysis engine, and is used to parse the time tags in the video task analysis request and transmit them to the intelligent video analysis engine; wherein, the time tags include the analysis frequency and the video acquisition time to be analyzed;

[0130] The algorithm request identification module is communicatively connected to both the request receiving module and the algorithm resource scheduling system. It is used to parse the algorithm tag in the video task analysis request and transmit it to the algorithm resource scheduling system. The algorithm tag is used to identify the algorithm type corresponding to the video task analysis request.

[0131] The algorithm resource scheduling system is communicatively connected to both the request analysis system and the intelligent video analysis engine. It is used to obtain a target scene algorithm identifier, determine whether the target scene algorithm identifier matches the algorithm tag of the video task analysis request, and if they match, generate an algorithm scheduling task list. It is also used to extract a target algorithm module from preset basic algorithm modules based on the algorithm scheduling task list. The target scene algorithm identifier refers to a scene algorithm identifier associated with the target image stream, and the algorithm type corresponding to the target algorithm module is consistent with the algorithm type identified by the algorithm tag. The basic algorithm modules are configured in a one-to-one correspondence with the algorithm tags.

[0132] The intelligent video analysis engine is communicatively connected to the request analysis system, the video information preprocessing system, and the algorithm resource scheduling system. It is used to read the structured tag stream, extract video information acquisition endpoints from the structured tag stream, and perform point-to-point matching between the extracted video information acquisition endpoints and the video information acquisition endpoints to be analyzed in the content request tags. It is also used to perform preliminary filtering of the original video information based on the point-to-point matching results to select target video information; to extract corresponding video segments from the target video information based on the analysis frequency in the time tags; to decode the extracted video segments to obtain the image stream to be analyzed; and to perform video acquisition time matching of the image stream to be analyzed based on the video acquisition time in the time tags to select the target image stream.

[0133] The intelligent video analysis engine is also used to call the target algorithm module to analyze and process the target image stream and generate image frame analysis results; wherein, the image frame analysis results include user identity tags and event analysis results, and the event analysis results include event images, event occurrence time, event occurrence location, event type, event confidence level, and target area coordinates;

[0134] The unified data sharing and exchange system is communicatively connected to the intelligent video analysis engine and is used to construct an index code for the image frame analysis results and store it in association with the image frame analysis results; wherein, the index code is set to correspond to the user identity tag in the video task analysis request;

[0135] The business application microservice system is connected to the unified data sharing and exchange system and the user terminal respectively. It is used to pre-store user identity tags and push the image frame analysis results to the corresponding user terminal according to the user identity tag corresponding to the index code.

[0136] It should be noted that the target video information refers to the initially filtered raw video information, which corresponds to the video information acquisition endpoint in the video source tag and matches the video information acquisition endpoint in the content request tag; the image stream to be analyzed refers to the initially filtered image stream, which is the image obtained after decoding the video segments, and these video segments are video segments extracted based on the analysis frequency in the time tag; the target image stream refers to the image stream that has been filtered again, and the video acquisition time in the video source tag matches the time tag of the video task analysis request.

[0137] Specifically, the algorithm tags for the video task analysis requests include tags for motor vehicle illegal parking analysis requests, non-motor vehicle illegal parking analysis requests, storefront operation analysis requests, road water accumulation analysis requests, exposed loess analysis requests, exposed garbage detection requests, crowd gathering detection requests, and people falling information requests.

[0138] Furthermore, the algorithm resource scheduling system also has functions such as external algorithm import, algorithm release, algorithm update, algorithm download, algorithm configuration, and algorithm open access. It is mainly designed for city-level application scenarios to realize the full-process management and configuration of various algorithms.

[0139] The business application microservices are developed based on a unified environment and framework, enabling city-level intelligent applications that cross departments, levels, and regions, such as street surface improvement, construction site violation management, and water area management.

[0140] The aforementioned microservice system enables users to quickly build the application capabilities they need, while showcasing the platform's empowering effects. Analysis results are pushed to a message queue in real time via microservices. Users subscribe to message topics in the queue based on their assigned business and scenario to receive the real-time analysis results. After subscribing to a message topic, the system processes the analysis results from a business perspective, combining the received message content with various standards for the business scenario, ultimately producing analysis results that strictly meet the needs of the business scenario.

[0141] It should be noted that, given the current state of video surveillance system construction in various departments, a large-scale city-level intelligent video analysis platform is a complex systems engineering project. There is no readily available experience to replicate, whether it's cross-platform integration, cross-organizational data fusion, cross-domain application incubation, or supporting management systems and mechanisms. This invention adopts a shared video and computing resource approach to integrate resources, fully utilize existing infrastructure, reduce redundant construction, and maximize the value of surveillance video. Through the construction of a large-scale city-level intelligent video analysis cluster platform, it shares monitoring resources from various commissions and departments, adopts a microservice approach to meet the regulatory requirements of each commission and department, and upgrades the monitoring system intelligently by maximizing the utilization of existing infrastructure.

[0142] Example 5

[0143] Based on the above embodiments, this embodiment presents another city-level large-scale intelligent video analytics cluster architecture;

[0144] Specifically, the unified data sharing and exchange system is also used to: generate response results for corresponding video task analysis requests based on the correlation between the image frame analysis results; wherein, the response results refer to the in-depth analysis results of one or more sets of video segments;

[0145] It is understandable that spatiotemporal information collision analysis can also be conducted on large-scale city-level events. Through global and real-time data analysis and judgment of the entire city, public resources can be effectively allocated and used, defects in city operation can be continuously discovered and corrected, some key contradictions in urban development that cannot be completely resolved by existing methods can be solved, and a management mechanism of "speaking with data, making decisions with data, managing with data, and innovating with data" can be established.

[0146] It is understood that the unified data sharing and exchange system will standardize and archive the image frame analysis results output by the intelligent video analysis engine and build an index, so as to facilitate subscription and push to various user terminals and maximize the use of event analysis results to meet the various needs of various user terminals.

[0147] Specifically, the analysis results of standardized archiving include, but are not limited to: image frame analysis results for public security, urban management, traffic, unified network management, environmental protection, water conservancy, etc.

[0148] Example 6

[0149] It should be noted that the system is built on the national supercomputing domestic DCU computing platform, and an intelligent video information analysis platform is deployed in the computing center. The intelligent video information analysis platform includes several parts such as a video access management system, an intelligent video analysis engine, a unified data sharing and exchange subsystem, business application microservices, and a supercomputing center resource management system.

[0150] It is understood that the intelligent video analysis engine, as the video parsing engine of the city-level large-scale intelligent video analysis cluster architecture, is the core system required for the execution of video parsing tasks. Its core function is to enable or disable a certain analysis of a certain video stream, provide the platform with the necessary video time-sharing, DCU intelligent video parsing, and visual computing scheduling capabilities, and is responsible for the identification and analysis of target image streams, using DCU computing resources to complete video parsing tasks, and realizing reasoning of events and structured results.

[0151] Therefore, based on the above embodiments, this embodiment also provides a specific implementation of an intelligent video analysis engine;

[0152] As attached Figure 4 As shown, the intelligent video analysis engine includes a video stream time-division calling module, a DCU hardware decoding module, a DCU intelligent video analysis module, and a visual computing task scheduling module.

[0153] (1) The video stream time-sharing invocation module includes a video time-sharing invocation unit, a strategy dynamic configuration unit, an image dynamic distribution unit, etc.;

[0154] 1) As attached Figure 2 As shown, the video time-division calling unit is used to call the domestic accelerator card DCU of the computing power center to perform point matching on the extracted video information acquisition endpoints; it is also used to call the domestic accelerator card DCU of the computing power center to perform preliminary screening of the original video information based on the point matching results, and to select target video information; it is also used to call the domestic accelerator card DCU of the computing power center to extract corresponding video segments from the target video information based on the analysis frequency in the time tag; it is also used to call the DCU hardware decoding module to decode the extracted video segments to obtain the image stream to be analyzed; and it is also used to call the domestic accelerator card DCU of the computing power center to perform video acquisition time matching on the image stream to be analyzed, and to select the target image stream from the image stream to be analyzed.

[0155] For city-level video intelligent analysis application events, this embodiment adopts a time-sharing call optimization strategy. The user terminal dynamically adjusts the analysis frequency of such events and generates corresponding video task analysis requests based on the frequency of event occurrence, thereby reducing the amount of detection tasks while ensuring the event detection effect.

[0156] 2) The image dynamic distribution unit is used to distribute target image streams that meet the analysis requests of various video tasks;

[0157] 3) The strategy dynamic configuration unit is used to support the dynamic adjustment of the real-time frame capture strategy and provide different real-time frame capture strategies according to the video task analysis requests sent by each user terminal.

[0158] (2) The DCU hardware decoding module includes a decoding multiplexing unit, a dynamic expansion and contraction unit, and a DCU decoding optimization unit. This embodiment is adapted for domestic computing centers. It calls DCU resources and utilizes the DCU hardware decoding module to decode video segments extracted from the original video information, which greatly improves the utilization efficiency of computing resources and the decoding rate.

[0159] 1) Implement a decoding image multiplexing strategy using a decoding multiplexing unit.

[0160] When user requests are diverse and multiple video task analysis requests occur within the same time period, and the video information required by multiple video task analysis requests contains the same video segment, in order to improve decoding efficiency, a decoding image multiplexing strategy is configured through a decoding multiplexing unit. When multiple video task analysis requests require the same video segment within the same time period, the purpose of multiple video task analysis requests can be satisfied by decoding once, thereby reducing the decoding workload and improving the decoding resource utilization rate.

[0161] 2) Utilize dynamic scaling units to expand and shrink the occupied hardware resources.

[0162] As attached Figure 5 As shown, the decoding concurrency corresponding to multiple video task analysis requests within the same time period is determined. When the decoding concurrency is greater than a first threshold, the hardware resources occupied by the DCU hardware decoding are dynamically expanded; when the decoding concurrency is less than a second threshold, the hardware resources occupied by the DCU hardware decoding are dynamically reduced. Based on the dynamically expanded or dynamically reduced hardware resources, the extracted video segments are decoded to obtain the image stream to be analyzed; wherein, the first threshold is greater than the second threshold.

[0163] Therefore, this embodiment can determine the decoding concurrency of different business scenarios according to the video analysis needs of each user terminal, so as to achieve the rational utilization of resources;

[0164] 3) The DCU decoding optimization unit has been adapted and optimized for use on domestic computing centers, supporting not only CPU decoding but also the use of DCU resources for decoding.

[0165] It should be noted that most existing deep learning frameworks currently use NVIDIA GPUs as parallel computing units, which means that existing deep learning algorithm models cannot directly call domestically produced DCU accelerator cards in computing centers for parallel computing.

[0166] Therefore, this embodiment analyzes the parallel computing process of deep learning framework operators step by step, locates the highly intensive computing part in the operator source code, and combines the hardware characteristics of domestic computing centers and the platform uses the HIP (Heterogeneous-compute Interface for Portability) heterogeneous programming model under the ROCm framework to map the intensive computing tasks to the DCU accelerator card.

[0167] Currently, the technology of domestic supercomputing centers is quite mature. The domestic supercomputing platform adopts a heterogeneous acceleration architecture of domestic x86 CPU + DCU. In the early stage of construction, factors such as compatibility and security were considered. The application development basic environment provided consists of programming models, compilation systems, and parallel development and runtime environments, including compilers, function libraries, parallel programming environments, and heterogeneous programming environments. It supports MPI-1 / MPI-2 / MPI-3 standards, supports OpenMP, and is compatible with OpenCL and CUDA environments. The compiler supports standard C / C++, Fortran, and other languages.

[0168] The main DCU architecture adaptation work of this invention mainly includes DCU video decoding, DCU inference performance optimization, and deep learning algorithm porting. Among them, the DCU contains a dedicated video decoding chip. Using the DCU to implement video decoding can significantly improve the utilization rate of supercomputing center computing resources.

[0169] It should be noted that when the DCU hardware decoding module decodes the extracted video segments to obtain the image stream to be analyzed, it performs the following operations: obtaining information such as the IP address, username, and device manufacturer of the accessed video stream through national standard protocols or vendor SDKs; or, obtaining RTSP bitstreams using the GB28181 protocol or by directly connecting to the camera, with the main bitstream format being H264 or H265 packets; or, selecting a multimedia framework based on ffmpeg and using multiple codecs such as x264 and x265 to achieve real-time decoding of H264 and H265 video streams to obtain video image frames.

[0170] (3) The DCU intelligent video analysis module includes an offline video analysis unit and a real-time video analysis unit, etc., which are used to call the target algorithm module to analyze and process the target image stream and generate image frame analysis results;

[0171] The deep learning algorithm has been ported and adapted to domestic computing power centers, supporting functions such as large-scale concurrent parsing of real-time video and speed-up parsing of offline video formats.

[0172] (4) The visual computing task scheduling module includes a computing scheduling unit, an offline task scheduling unit, and a real-time task scheduling unit. In response to the characteristics of large-scale concurrent parsing tasks at the city level, the visual computing task scheduling module dynamically calls hardware resources according to the amount of concurrent tasks, which can achieve a smooth upgrade of the number of concurrent parsing video channels.

[0173] 1) The computing scheduling unit dynamically adjusts the computing resources used based on the number of concurrent video streams in real time;

[0174] 2) Offline task scheduling unit, which supports the parsing of offline videos. In response to emergencies, it can schedule different resources for offline video parsing as needed and dynamically adjust the parsing rate according to the resources.

[0175] 3) The real-time task scheduling unit adjusts computing resources and schedules tasks in real time according to the video analysis needs of each user terminal.

[0176] In other embodiments, a video access management system is set up between the video information acquisition terminal and the video information preprocessing system of the intelligent video information analysis platform. The intelligent video information analysis platform relies on the video access management system to access massive video point resources, utilizes the large-scale computing and storage capabilities provided by the supercomputing center resource management system, and uses the DCU intelligent video parsing engine to detect and analyze targets and events in videos / images. The identified events and structured results are used as image frame analysis results and are aggregated into a unified data sharing and exchange system for spatiotemporal collision analysis and index storage. Finally, each business application microservice is responsible for the push management of events according to the subscription status of each user terminal (business unit).

[0177] The video access management system is responsible for the access management of multi-source monitoring data from various business units in the city, and provides access and management services for various video resources, including local video resources and networked video resources. Local video resources are mainly accessed through various protocols such as device SDK, GB / T28181, and ONVIF. Networked video resources are accessed through platform cascading, and support network access through national standard protocols or third-party platform SDKs.

[0178] Furthermore, the basic peripheral services upon which the intelligent video information analysis platform relies include DNS service, image storage service, structured storage, service registry, configuration center, big data storage, and message queue; the combination of image storage service, structured storage, and big data storage provides the necessary storage services for the platform.

[0179] As the largest and most distinctive data format in intelligent analytics platforms, images are the subject of dedicated image storage services, which are optimized to address specific image storage needs. Structured data storage primarily targets online transaction issues, storing limited-scale, business-related structured data. Big data storage addresses the massive storage challenges of big data scenarios, providing long-term storage capabilities for large datasets. Image storage and structured data storage primarily focus on storing important data within a certain timeframe, without addressing long-term storage requirements.

[0180] In other embodiments, the intelligent video information analysis platform is also configured with a resource management and scheduling system, specifically including a basic resource scheduling and management module and a service resource scheduling and management module; it manages basic resources, algorithm resources and service resources according to the needs of each user terminal (business unit), and realizes resource allocation for large-scale video analysis at the city level through scheduling management, thereby achieving efficient utilization of various resources;

[0181] The basic resource scheduling and management module is used to implement functions such as video resource management, computing resource management, intelligent scheduling, and storage resource management. The video resource management function allows administrators to view basic information, catalog categories, and device status of video point resources connected to the intelligent video information analysis platform, facilitating access, management, and status monitoring of these resources. The computing resource management function enables unified access and management of underlying CPU computing resources, DCU visual computing resources, and storage resources.

[0182] The service resource scheduling and management module mainly provides basic service configuration management for the entire platform, specifically for implementing algorithm service management, push service management, gateway service management, cache service management, storage service management, and data service management.

[0183] Example 7

[0184] This embodiment uses the XX traffic platform as the user terminal, the XX traffic management unit as the user identity tag, and the video task analysis request as the detection of illegal parking events in temporary parking spaces at night as an example to illustrate the city-level large-scale intelligent video analysis method and cluster architecture:

[0185] Preparation phase: Infrastructure construction, through network transformation to aggregate existing monitoring data to the computing power center of the cluster service, such as community monitoring, high-point monitoring, public security monitoring, key location monitoring, etc., to access the unified computing power center;

[0186] The analysis phase includes video information preprocessing, request analysis, algorithm resource scheduling, video analysis, and event reporting.

[0187] Video information preprocessing process: Obtain the video source tags and new tags of the original video information, generate a structured tag stream and store it; wherein, the original video information comes from various community monitoring, high-point monitoring, traffic management monitoring, public security monitoring, key location monitoring, etc., the video source tags include video acquisition time and video information acquisition endpoint, the new tags include scene algorithm identifiers, and the scene identifiers include motor vehicle illegal parking scene algorithm identifiers, etc.

[0188] Request analysis process: The user generates a video task analysis request (for illegal parking events) and transmits it to the request analysis system; the request analysis system parses out the user identity tag, content request tag, time tag, and algorithm tag from the video task analysis request;

[0189] The user identity tag includes traffic management units, etc.; the content request tag includes the endpoint of the video information to be analyzed, and the unit to which the video of the endpoint belongs is the traffic management and monitoring department; the time tag includes the analysis frequency and the video collection time to be analyzed. The analysis frequency is set according to the frequency of event requirements, such as once every 5 minutes, and the video collection time to be analyzed is from 7:00 am to 8:00 pm; the algorithm tag is the motor vehicle illegal parking analysis request tag.

[0190] Algorithm resource scheduling process: Obtain the target scene algorithm identifier, which is the algorithm identifier for the illegal parking scene of motor vehicles; determine whether the target scene algorithm identifier matches the algorithm tag of the video task analysis request; if they match, generate an algorithm scheduling task order.

[0191] According to the algorithm scheduling task list, a target algorithm module is extracted from the preset basic algorithm module; the target algorithm module is configured with an illegal parking detection and recognition algorithm.

[0192] Video analysis process: The structured tag stream is read, the video information acquisition endpoints in the structured tag stream are extracted, and the extracted video information acquisition endpoints are matched with the video information acquisition endpoints to be analyzed in the content request tag; based on the matching results, the original video information is initially filtered to select target video information; wherein, the points corresponding to the target video information are the video information acquisition endpoints to be analyzed belonging to traffic management monitoring.

[0193] Based on the analysis frequency in the time tag, the corresponding video segment is extracted from the target video information; for example, based on the analysis frequency of once every 5 minutes, a 2-minute video segment is extracted every 5 minutes.

[0194] The extracted video segments are decoded to obtain the image stream to be analyzed; the video capture time of the image stream to be analyzed is matched based on the capture time of the video to be analyzed in the time tag, and the target image stream is selected from the image stream to be analyzed; wherein, the video capture time corresponding to the target image stream is from 7:00 am to 8:00 pm;

[0195] Based on the target algorithm module, illegal parking events in temporary parking spaces at night are detected. If an illegal parking event in a temporary parking space at night is detected, an image frame analysis result is generated. The image frame analysis result includes the illegal parking event image (any image in the target image stream), the illegal parking event sending time (the video acquisition time corresponding to the target image stream), the event location (the corresponding video information acquisition endpoint), the event type (illegal parking event in a temporary parking space at night), the event confidence (the probability that the illegal parking event image is judged as an illegal parking event in a temporary parking space at night; since the point corresponding to the target video information belongs to traffic management monitoring, the event confidence is set to high, such as 95%), and the target area coordinates (pixel coordinates on the image).

[0196] The image frame analysis results are stored in a unified data sharing and exchange system. The unified data sharing and exchange system constructs an index code for the image frame analysis results and stores it in association with the corresponding image frame analysis results.

[0197] Event Reporting: Since the index code is set to correspond with the XX traffic management unit (user identity tag), the business application microservice system uses a microservice approach to connect with the XX traffic platform. Based on the index code, it quickly reads the required image frame analysis results and reports the image frame analysis results and other information to the XX traffic platform according to the subscription requirements.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A city-level large-scale intelligent video analysis method, characterized in that, Includes the following steps; Step 1: Obtain the video source tags and new tags of the original video information, and generate and store a structured tag stream based on the video source tags and the new tags; The original video information comes from the corresponding video information acquisition terminal, the video source tag includes the video acquisition time and the video information acquisition terminal position, and the new tag includes a scene algorithm identifier, which is used to identify the algorithm information pre-configured for the original video information. Step 2: Determine if a video task analysis request has been received from the user's client. If so, then parse out the user identity tag, content request tag, time tag and algorithm tag in the video task analysis request, and proceed to step 3; The content request tag includes the endpoint of the video information to be analyzed, the time tag includes the analysis frequency and the video acquisition time to be analyzed, and the algorithm tag is used to identify the algorithm type corresponding to the video task analysis request. Step 3: Read the structured tag stream, extract the video information acquisition endpoints from the structured tag stream, and perform point matching between the extracted video information acquisition endpoints and the video information acquisition endpoints to be analyzed in the content request tag; Based on the location matching results, the original video information is initially filtered to select the target video information; Step 4: Extract the corresponding video segments from the target video information based on the analysis frequency in the time tags; The extracted video segments are decoded to obtain the image stream to be analyzed; Step 5: Match the video capture time of the image stream to be analyzed based on the video capture time in the time tag, and filter out the target image stream from the image stream to be analyzed; Step 6: Obtain the target scene algorithm identifier, and determine whether the target scene algorithm identifier matches the algorithm tag of the video task analysis request. If they match, generate an algorithm scheduling task sheet. Wherein, the target scene algorithm identifier refers to the scene algorithm identifier associated with the target image stream; Step 7: According to the algorithm scheduling task list, extract the target algorithm module from the preset basic algorithm modules; wherein, the basic algorithm modules are set in a one-to-one correspondence with the algorithm tags; The target algorithm module is invoked to analyze and process the target image stream, generating image frame analysis results; The image frame analysis results include user identity tags and event analysis results. The event analysis results include event images, event occurrence time, event occurrence location, event type, event confidence level, and target area coordinates. Step 8: Construct an index code for the image frame analysis results and store it in association with the image frame analysis results; The index code is set to correspond to the user identity tag in the video task analysis request; Step 9: Based on the user identity tag corresponding to the index code, push the image frame analysis results to the corresponding user terminal.

2. The city-level large-scale intelligent video analysis method according to claim 1, characterized in that, Step 7 includes the following steps: Step 701: Read the target image stream corresponding to the video task analysis request; Step 702: Parse the algorithm tags in the algorithm scheduling task sheet, and perform algorithm type matching between the parsed algorithm tags and the algorithm tags corresponding to the basic algorithm modules; Based on the algorithm type matching results, the target algorithm module is extracted from the preset basic algorithm modules; Step 703: Based on the target algorithm module, perform algorithmic reasoning on the read target image stream to obtain image frame analysis results; The image frame analysis results are JSON format string data, including user identity tags, event images, event time, event location, event type, event confidence level, and target area coordinates.

3. The city-level large-scale intelligent video analysis method according to claim 1, characterized in that, Before pushing the image frame analysis results to the corresponding user terminal in step 9, the following is also performed: Based on the correlation between the image frame analysis results, a response result for the corresponding video task analysis request is generated; The response result refers to the in-depth analysis results of one or more video segments.

4. A city-level large-scale intelligent video analytics cluster architecture, characterized in that: It includes a video information acquisition terminal, a user terminal, and an intelligent video information analysis platform. The intelligent video information analysis platform includes a video information preprocessing system, a request analysis system, an algorithm resource scheduling system, an algorithm module, an intelligent video analysis engine, a unified data sharing and exchange system, and a business application microservice system. The video information preprocessing system includes a video source tag management module and a tag stream management module. The request analysis system includes a request receiving module, a content request recognition module, a time-sharing invocation module, and an algorithm request recognition module. The video information acquisition terminal is communicatively connected to the video information preprocessing system, and is used to acquire video information in real time and transmit the acquired raw video information to the video information preprocessing system. The video source tag management module is used to obtain the video source tag of the original video information; wherein, the video source tag includes the video capture time and the video information capture endpoint. The tag stream management module is used to acquire new tags for the original video information, generate a structured tag stream based on the video source tags and the new tags, and transmit the structured tag stream to the intelligent video analysis engine; wherein, the new tags include scene algorithm identifiers, which are used to identify algorithm information pre-configured for the original video information; The user terminal is connected to the request analysis system and is used to generate different video task analysis requests and transmit them to the request analysis system. The request receiving module is communicatively connected to both the user terminal and the intelligent video analysis engine. It is used to determine whether a video task analysis request has been received from the user terminal. If so, it parses out the user identity tag in the video task analysis request and transmits it to the intelligent video analysis engine. It is also used to transmit the received video task analysis request to the content request recognition module, the time-sharing calling module, and the algorithm request recognition module, respectively. The content request recognition module is communicatively connected to both the request receiving module and the intelligent video analysis engine, and is used to parse the content request tag in the video task analysis request and transmit it to the intelligent video analysis engine; wherein, the content request tag includes the endpoint bit of the video information to be analyzed; The time-sharing invocation module is communicatively connected to both the request receiving module and the intelligent video analysis engine, and is used to parse the time tags in the video task analysis request and transmit them to the intelligent video analysis engine; wherein, the time tags include the analysis frequency and the video acquisition time to be analyzed; The algorithm request identification module is communicatively connected to both the request receiving module and the algorithm resource scheduling system. It is used to parse the algorithm tag in the video task analysis request and transmit it to the algorithm resource scheduling system. The algorithm tag is used to identify the algorithm type corresponding to the video task analysis request. The algorithm resource scheduling system is communicatively connected to the request analysis system and the intelligent video analysis engine, respectively. It is used to obtain the target scene algorithm identifier, determine whether the target scene algorithm identifier matches the algorithm tag of the video task analysis request, and generate an algorithm scheduling task list if they match. It is also used to extract the target algorithm module from the preset basic algorithm module according to the algorithm scheduling task list. The target scene algorithm identifier refers to the scene algorithm identifier associated with the target image stream. The intelligent video analysis engine is communicatively connected to the request analysis system, the video information preprocessing system, and the algorithm resource scheduling system. It is used to read the structured tag stream, extract video information acquisition endpoints from the structured tag stream, and perform point-to-point matching between the extracted video information acquisition endpoints and the video information acquisition endpoints to be analyzed in the content request tags. It is also used to perform preliminary filtering of the original video information based on the point-to-point matching results to select target video information; to extract corresponding video segments from the target video information based on the analysis frequency in the time tags; to decode the extracted video segments to obtain the image stream to be analyzed; and to perform video acquisition time matching of the image stream to be analyzed based on the video acquisition time in the time tags to select the target image stream. The intelligent video analysis engine is also used to call the target algorithm module to analyze and process the target image stream and generate image frame analysis results; wherein, the image frame analysis results include user identity tags and event analysis results, and the event analysis results include event images, event occurrence time, event occurrence location, event type, event confidence level, and target area coordinates; The unified data sharing and exchange system is communicatively connected to the intelligent video analysis engine and is used to construct an index code for the image frame analysis results and store it in association with the image frame analysis results; wherein, the index code is set to correspond to the user identity tag in the video task analysis request; The business application microservice system is connected to the unified data sharing and exchange system and the user terminal respectively. It is used to pre-store user identity tags and push the image frame analysis results to the corresponding user terminal according to the user identity tag corresponding to the index code.

5. The city-level large-scale intelligent video analytics cluster architecture according to claim 4, characterized in that, The unified data sharing and exchange system is also used for: Based on the correlation between the image frame analysis results, a response result for the corresponding video task analysis request is generated; The response result refers to the in-depth analysis results of one or more video segments.

6. The city-level large-scale intelligent video analytics cluster architecture according to claim 4, characterized in that: The algorithm tags for the video task analysis requests include tags for motor vehicle illegal parking analysis requests, non-motor vehicle illegal parking analysis requests, storefront operation analysis requests, road water accumulation analysis requests, exposed loess analysis requests, exposed garbage detection requests, crowd gathering detection requests, and people falling information requests.

Images