Passenger flow early warning analysis method, electronic equipment and storage medium

Abstract

The invention provides a passenger flow early warning analysis method, electronic equipment and a storage medium, and the method comprises the steps: obtaining the information needed by passenger flow early warning, obtaining the station passenger flow data of a target station and the region monitoring video of a to-be-monitored region under the time information according to the information of the target station and the information of the to-be-monitored region, and obtaining target passenger flow early warning information according to the target passenger flow abnormity category, performing passenger flow abnormity detection on the area monitoring video according to the target passenger flow abnormity category to obtain an abnormity detection result of the target passenger flow abnormity category, and performing passenger flow early warning on the target passenger flow early warning information by adopting a natural language large model according to the station passenger flow data, the target passenger flow early warning information and the abnormity detection result. And generating an early warning analysis result for the target station. According to the scheme, natural language interaction is supported, not only can statistical analysis be performed, but also video information can be processed, the early warning analysis result of the target station is generated by using the AI large model, and multi-dimensional analysis is realized.

Description

Technical Field

[0001] The present application relates to the field of rail transit technology, and in particular to a passenger flow warning analysis method, electronic equipment and storage medium. Background Art

[0002] Currently, common reporting products are committed to providing full-process support from data connection, modeling to report production and presentation. Some reporting products focus on report design and server deployment, support connection with multiple data sources, and realize report development and publishing through rich functional modules. Some reporting products use browsers to directly design and view reports, provide visualization components such as charts, tables and cross-tables, and improve data intuitiveness and decision-making efficiency.

[0003] However, in the field of passenger flow analysis, when using the above-mentioned reporting products for passenger flow analysis, users are usually required to be familiar with operations such as table layout, parameter configuration, and data modeling. The usage threshold is high and it is difficult to get started. In addition, it can only process numerical information to display numerical statistical information, and cannot process multimodal information such as video and provide risk warnings. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a passenger flow warning analysis method, an electronic device and a storage medium to solve the problem that when using current reporting products for passenger flow analysis, users are usually required to be familiar with operations such as table layout, parameter configuration and data modeling, the usage threshold is high, it is difficult to get started, and it can only process numerical information to give statistical charts, but cannot process multimodal information such as video.

[0005] In a first aspect, an embodiment of the present application provides a passenger flow early warning analysis method, comprising:

[0006] Acquire information required for passenger flow warning, the information required for passenger flow warning includes: information of target station, information of the area to be monitored in the target station, target passenger flow abnormality category and time information;

[0007] According to the information of the target station and the information of the area to be monitored, obtaining the station passenger flow data of the target station under the time information and the regional monitoring video of the area to be monitored;

[0008] According to the target passenger flow abnormality category, obtaining target passenger flow warning information;

[0009] According to the target passenger flow abnormality category, performing passenger flow abnormality detection on the regional surveillance video to obtain an abnormality detection result of the target passenger flow abnormality category;

[0010] Based on the station passenger flow data, the target passenger flow warning information and the abnormal detection results, a natural language large model is used to generate a warning analysis result for the target station.

[0011] In an optional implementation manner, the obtaining of information required for passenger flow warning includes:

[0012] Acquire the target station information and the time information input through the text input box;

[0013] In response to the information input operation of the target station, displaying the area selection box and the passenger flow abnormality selection box of the target station;

[0014] The information of the area to be monitored and the target passenger flow abnormality category inputted through the area selection box and the passenger flow abnormality selection box are obtained respectively.

[0015] In an optional implementation, performing passenger flow anomaly detection on the regional surveillance video according to the target passenger flow anomaly category to obtain an anomaly detection result of the target passenger flow anomaly category includes:

[0016] generating a target prompt word according to the target passenger flow abnormality category;

[0017] According to the target prompt word, a multimodal large model is used to perform passenger flow anomaly detection on the regional surveillance video to obtain the anomaly detection result.

[0018] In an optional implementation, the method of performing passenger flow anomaly detection on the regional surveillance video using a multimodal large model according to the target prompt word to obtain the anomaly detection result includes:

[0019] Performing frame extraction processing on the regional surveillance video to obtain multiple surveillance video frames;

[0020] According to the target prompt word, the multimodal large model is used to perform passenger flow anomaly detection on the multiple surveillance video frames to obtain the anomaly detection result.

[0021] In an optional implementation, generating a target prompt word according to the target passenger flow abnormality category includes:

[0022] Generate the target prompt word according to the target passenger flow abnormality category and a preset detection result template;

[0023] The method of performing passenger flow anomaly detection on the plurality of surveillance video frames using the multimodal large model according to the target prompt word to obtain the anomaly detection result includes:

[0024] According to the target prompt word, the multimodal large model is used to perform passenger flow anomaly detection on the multiple surveillance video frames to obtain anomaly detection results of the target passenger flow anomaly category arranged according to the preset detection result template.

[0025] In an optional implementation, generating a target prompt word according to the target passenger flow abnormality category includes:

[0026] Loading the prompt word template corresponding to the multimodal large model;

[0027] According to the target passenger flow abnormality category, the prompt word template is filled to generate the target prompt word.

[0028] In an optional implementation manner, the time information includes: date information, and a time unit under the date information, and the method further includes:

[0029] The station passenger flow data is grouped and summarized according to the date information and the time unit, and a passenger flow analysis chart corresponding to the time unit under the date information is generated.

[0030] In an optional embodiment, the method further comprises:

[0031] Load the preset station passenger flow report template;

[0032] According to the early warning analysis result and the passenger flow analysis chart, the station passenger flow report template is used to generate a station passenger flow report for the target station.

[0033] In a second aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to execute any one of the methods described in the first aspect.

[0034] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the first aspects is executed.

[0035] The present application provides a passenger flow warning analysis method, electronic device and storage medium, wherein the method includes: obtaining the information required for passenger flow warning, the information required for passenger flow warning includes: information of the target station, information of the area to be monitored in the target station, target passenger flow abnormality category and time information, obtaining the station passenger flow data of the target station under the time information and the regional monitoring video of the area to be monitored according to the information of the target station and the information of the area to be monitored, obtaining the target passenger flow warning information according to the target passenger flow abnormality category, performing passenger flow abnormality detection on the regional monitoring video according to the target passenger flow abnormality category, obtaining the abnormality detection result of the target passenger flow abnormality category, and generating the warning analysis result for the target station according to the station passenger flow data, the target passenger flow warning information and the abnormality detection result, using the natural language large model. This solution supports natural language interaction, can not only perform statistical analysis, but also process video information, and uses the AI ​​large model to generate the warning analysis result of the target station, realizing multi-dimensional analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 1 ;

[0038] Figure 2 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 2 ;

[0039] Figure 3 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 3 ;

[0040] Figure 4 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 4 ;

[0041] Figure 5 A schematic diagram of a front-end interface provided in an embodiment of the present application;

[0042] Figure 6 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 5 ;

[0043] Figure 7A schematic diagram of the structure of a passenger flow warning and analysis device provided in an embodiment of the present application;

[0044] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0046] Before introducing the solution of this application, the professional terms involved are first explained:

[0047] LSTM: Long Short-Term Memory network is an improved recurrent neural network (RNN). By introducing memory units and gate mechanisms, it can efficiently capture and retain important information in long time series. It is suitable for tasks such as time series prediction and natural language processing.

[0048] Database: A system that electronically stores, manages and organizes data for efficient access, updating and maintenance of large amounts of structured or unstructured data.

[0049] Natural Language Big Model: It is a large-scale neural network based on deep learning. It can understand and generate natural language by learning massive text data. It is used for various tasks such as question answering, translation, and text generation.

[0050] Multimodal big model: A multimodal big model is a deep learning model that integrates multiple data types (such as text, images, audio, etc.). It can achieve cross-modal understanding and generation and is widely used in intelligent search, image and text generation and other fields.

[0051] Prompt words: Prompt words are input texts designed for natural language models. They aim to guide the model to generate specific content or complete specific tasks. They are a key means to enhance model capabilities and achieve multi-functional applications.

[0052] At present, in the field of passenger flow analysis, when using existing reporting products for passenger flow analysis, users are usually required to be familiar with operations such as table layout, parameter configuration, and data modeling, and understand the use of data structure and visualization tools. The threshold for using the product is high, and it is difficult for non-technical users or users without programming background to get started, and the system deployment and integration are difficult. In addition, existing data reporting products can only process numerical information (structured data) to display numerical statistical information, and the in-depth analysis of its results depends on the interpretation of professionals. In addition, it is impossible to process multimodal information such as video and provide risk warnings.

[0053] Based on this, this application does not require users to master complex table layout, parameter configuration or data modeling skills, supports natural language interaction, and the system automatically recognizes user intentions. It can not only perform statistical analysis, but also process video information (unstructured data) and use AI large models to generate early warning analysis results for the target station, realizing multi-dimensional analysis.

[0054] Figure 1 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 1 The execution subject of this embodiment can be an electronic device, such as a mobile phone, a tablet computer, a desktop computer, etc.

[0055] like Figure 1 As shown, the method may include:

[0056] S101. Obtain information required for passenger flow warning.

[0057] The information required for the passenger flow warning may be input by the user through the front-end interface, and the information required for the passenger flow warning includes: information of the target station, information of the area to be monitored in the target station, target passenger flow abnormality category and time information.

[0058] The target station is the station where passenger flow warning is to be carried out. The information of the target station can be the station identification of the target station. The target station includes multiple areas, such as the lobby area, the security area, and the entry and exit area. The area to be monitored is any area among the multiple areas. The information of the area to be monitored can be the area identification of the area to be monitored, such as the "lobby area".

[0059] The target passenger flow anomaly category is used to indicate the category of passenger flow anomaly detection for the monitored area. The number of target passenger flow anomaly categories includes but is not limited to one, and may include, for example, person falling, person walking in the opposite direction on an escalator, person gathering, etc.

[0060] The time information is the time for issuing passenger flow warning for the target station. The time information may include date information, or time information and time units under the date information. For example, the date information is January 1, 2024, and the time unit is hours, which means that passenger flow warning is issued for every hour on January 1, 2024.

[0061] S102. According to the information of the target station and the information of the area to be monitored, the station passenger flow data of the target station and the regional monitoring video of the area to be monitored under the time information are obtained.

[0062] According to the information of the target station, the station passenger flow data (passenger flow) of the target station under the time information is queried, and according to the information of the area to be monitored, the regional monitoring video of the area to be monitored under the time information is queried. For example, the regional monitoring video of the area to be monitored every hour on January 1, 2024 and the station passenger flow data of the target station every hour on January 1, 2024 are queried.

[0063] In some embodiments, the preset database stores the station passenger flow data of the target station at each time information, and the regional monitoring video of each area of ​​the target station, or the preset video storage system stores the regional monitoring video of each area of ​​each station at each time information. Therefore, according to the information of the target station and the information of the area to be monitored, the station passenger flow data of the target station at the time information and the regional monitoring video of the area to be monitored can be queried from the preset database, or according to the information of the area to be monitored, the regional monitoring video of the area to be monitored at the time information can be queried from the preset video storage system.

[0064] It is worth noting that based on the information of the target station and the information of the area to be monitored, a video file query request can be generated, and based on the video file query request, a preset video storage system or a preset database can be queried to obtain the regional monitoring video of the area to be monitored under the time information. It can also be checked whether the regional monitoring video is complete and available. Checking whether the video is available refers to determining whether the regional monitoring video can be parsed, played, and whether the format verification has passed, etc.

[0065] In one possible implementation, a regular expression is used to parse and identify the time information in the information required for passenger flow warning, and the jieba library is used to switch the information required for passenger flow warning into several words, and based on the locally stored information of each station, the information of each area, and the passenger flow abnormality category, the information of the target station, the information of the area to be monitored, and the target passenger flow abnormality category are searched in the segmented words, and then the locally stored structured query language (Structured Query Language, SQL) template is recorded, and the information of the target station is filled into the SQL template as a structured parameter to obtain a first SQL query statement for querying the passenger flow data of the target station under the time information, and the information of the area to be monitored is filled into the SQL template as a structured parameter to obtain a second SQL query statement for querying the regional monitoring video of the area to be monitored under the time information.

[0066] S103. Obtain target passenger flow warning information according to the target passenger flow abnormality category.

[0067] Among them, the preset database stores various passenger flow abnormality categories and the passenger flow warning information corresponding to each passenger flow abnormality category. The passenger flow warning information corresponding to each passenger flow abnormality category is used to indicate the handling measures for the passenger flow abnormality category. For example, the passenger flow warning information corresponding to a person falling can be "find medical staff nearby, and / or evacuate people nearby to prevent secondary trampling", and the passenger flow warning information corresponding to a person going backwards on an escalator can be "set up a sign prohibiting the escalator from going backwards, and / or add security personnel and guide personnel at the escalator to prevent the escalator from going backwards".

[0068] Therefore, according to the target passenger flow abnormality category, the passenger flow warning information corresponding to the target passenger flow abnormality category can be determined from a preset database as the target passenger flow warning information.

[0069] In some embodiments, the stored passenger flow warning information corresponding to each passenger flow abnormality category is loaded from a preset database, and the passenger flow warning information may be in JSON or YAML format. Then, according to the target passenger flow abnormality category, the applicability of the stored passenger flow warning information is verified to extract the passenger flow warning information corresponding to the target passenger flow abnormality category from the passenger flow warning information corresponding to each passenger flow abnormality category, and obtain the target passenger flow warning information. That is, according to the target passenger flow abnormality category, the stored passenger flow warning information is filtered and extracted, and the target passenger flow warning information matching the target station and time information is returned.

[0070] In some embodiments, since the passenger flow warning information corresponding to each passenger flow abnormality category stored in the preset database is manually input in advance, after determining the target passenger flow warning information, it is also possible to verify whether the passenger flow warning information corresponding to the target passenger flow abnormality category is complete. If it is complete, the passenger flow warning information corresponding to the target passenger flow abnormality category is determined to be the target passenger flow warning information.

[0071] S104: Perform passenger flow anomaly detection on the regional surveillance video according to the target passenger flow anomaly category to obtain an anomaly detection result of the target passenger flow anomaly category.

[0072] According to the target passenger flow anomaly category, the regional surveillance video is analyzed, and passenger flow anomaly detection is performed on the regional surveillance video to obtain anomaly detection results of the target passenger flow anomaly category, wherein the anomaly detection results are used to indicate whether there is a passenger flow anomaly for the target passenger flow anomaly category in the monitored area under the time information.

[0073] S105. Based on the station passenger flow data, target passenger flow warning information and anomaly detection results, a natural language big model is used to generate warning analysis results for the target station.

[0074] A large natural language model is used to analyze the station passenger flow data, target passenger flow warning information and anomaly detection results, and the warning analysis results for the target station are obtained, wherein the warning analysis results are used to indicate whether the target station or the monitored area may have passenger flow anomalies for the target passenger flow anomaly category when the real-time passenger flow data of the target station reaches the passenger flow data of the station at a future time and a time granularity corresponding to the time information, and to take the target passenger flow warning information as a processing measure when the passenger flow anomaly may exist.

[0075] It can be understood that if the anomaly detection result indicates that there is a passenger flow anomaly for the target passenger flow anomaly category in the monitored area under the time information, then the early warning analysis result indicates that at the future time and the time granularity corresponding to the time information, the target station or the monitored area may have a passenger flow anomaly for the target passenger flow anomaly category. If the anomaly detection result indicates that there is no passenger flow anomaly for the target passenger flow anomaly category in the monitored area under the time information, then the early warning analysis result indicates that at the time granularity corresponding to the time information, the target station or the monitored area may not have a passenger flow anomaly for the target passenger flow anomaly category.

[0076] For example, the time information is January 1, 2024, and the passenger flow data of the station is 1,000 people. The anomaly detection result indicates that there is a passenger flow anomaly for the target passenger flow anomaly category in the monitored area. The early warning analysis results for the target station include the future time (time after January 1, 2024) and the granularity of the day. If the real-time passenger flow data of the target station reaches 1,000 people, it is determined that there may be a passenger flow anomaly for the target passenger flow anomaly category in the monitored area. The target passenger flow early warning information can be taken in advance as a processing measure to prevent or minimize the impact of the passenger flow anomaly for the target passenger flow anomaly category. For example, if the target passenger flow anomaly category is a fall, the target passenger flow early warning information "find medical staff nearby, and / or evacuate people nearby to prevent secondary trampling" will be taken in advance.

[0077] It can be understood that if the anomaly detection result indicates that there is no passenger flow anomaly for the target passenger flow anomaly category in the monitored area, the early warning analysis result for the target station includes a time granularity of a day. If the real-time passenger flow data of the target station reaches 1,000 people, it is determined that there may be no passenger flow anomaly for the target passenger flow anomaly category in the monitored area.

[0078] In this embodiment, natural language interaction is supported, which can not only perform statistical analysis, but also process video information (unstructured data), and use the AI ​​big model to generate early warning analysis results for the target station, realizing multi-dimensional analysis.

[0079] Figure 2 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 2 ,like Figure 2 As shown, in an optional implementation, the above step S101, obtaining the information required for passenger flow warning, includes:

[0080] S201. Obtain the target station information and time information input through the text input box.

[0081] Among them, the front-end interface provides a text input box, and the user inputs the information and time information of the target station through the text input box, so that the information and time information of the target station are displayed in the text input box.

[0082] It is understandable that after the user enters information through the text input box, it is also possible to check whether the user input information is complete. If it is incomplete, a prompt to complete the information can be pushed to enable the user to further complete the input information through the text input box.

[0083] S202. In response to the information input operation of the target station, an area selection box and an abnormal passenger flow selection box of the target station are displayed.

[0084] Among them, the information input operation of the target station is used to input the target station into the text input box. The information input operation is an operation in which the user inputs the target station through the text input box.

[0085] In response to the information input operation of the target station, the area selection box and the passenger flow abnormality selection box of the target station are displayed, wherein the area selection box provides multiple area options of the target station, and the passenger flow abnormality selection box provides multiple passenger flow abnormality categories.

[0086] In some embodiments, the area options in the area selection box may be in the form of a drop-down menu, and the passenger flow anomaly categories in the passenger flow anomaly selection box may also be in the form of a drop-down menu.

[0087] S203: Acquire information of the area to be monitored and target passenger flow anomaly category inputted through the area selection box and the passenger flow anomaly selection box respectively.

[0088] The information of the area to be monitored inputted through the area selection box refers to the user selecting the area to be monitored option from a plurality of area options through the area selection box to input the information of the area to be monitored.

[0089] The target passenger flow abnormality category input through the passenger flow abnormality selection box refers to the user selecting the target passenger flow abnormality category from multiple passenger flow abnormality categories through the passenger flow abnormality selection box to input the target passenger flow abnormality category.

[0090] In this embodiment, natural language interaction is supported. Users only need to input query requirements through natural language in the text input box. There is no need to master complex table layout, parameter configuration or data modeling skills. The system will automatically identify user intentions, complete data queries, and present the results in the form of reports. In this way, even users with non-technical backgrounds can easily get started and quickly obtain the required information, thereby lowering the threshold for using data reports and providing users with a more intuitive and convenient operating experience.

[0091] Figure 3 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 3 ,like Figure 3 As shown, in an optional implementation, the above step S104, according to the target passenger flow abnormality category, performs passenger flow abnormality detection on the regional monitoring video to obtain the abnormality detection result of the target passenger flow abnormality category, which may include:

[0092] S301. Generate target prompt words according to the target passenger flow abnormality category.

[0093] S302: According to the target prompt word, a multimodal large model is used to perform passenger flow anomaly detection on the regional surveillance video to obtain anomaly detection results.

[0094] The target prompt word is used as the input of the multimodal large model, and the multimodal large model is used to perform passenger flow anomaly detection on the regional surveillance video to obtain the anomaly detection result of the target passenger flow anomaly category. The anomaly detection result is used to indicate whether there is a passenger flow anomaly for the target passenger flow anomaly category in the monitored area under the time information.

[0095] In an optional implementation, the above step S301, generating a target prompt word according to the target passenger flow abnormality category, may include:

[0096] Load the prompt word template corresponding to the multimodal large model;

[0097] According to the target passenger flow abnormality category, fill in the prompt word template and generate the target prompt word.

[0098] The prompt word template (Prompt template) corresponding to the multimodal large model is loaded, and the target passenger flow abnormality category is used as a template parameter to fill in the prompt word template to obtain the target prompt word.

[0099] For example, if the target passenger flow anomaly categories include crowds and people falling, the target prompt words are used to guide the multimodal large model to make the following judgments: whether there is crowd gathering and whether there is a person falling.

[0100] In this embodiment, a multimodal large model is used to process video information (unstructured data) to obtain anomaly detection results, thereby providing better services and empowering the business.

[0101] Figure 4 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 4 ,like Figure 4 As shown, in an optional implementation, the above step S302, based on the target prompt word, uses a multimodal large model to perform passenger flow anomaly detection on the regional surveillance video to obtain anomaly detection results, which may include:

[0102] S401: Perform frame extraction processing on the regional monitoring video to obtain multiple monitoring video frames.

[0103] Frame extraction refers to extracting a number of frames at certain intervals in a video. Frame extraction is performed on the regional surveillance video to obtain multiple surveillance video frames.

[0104] S402: According to the target prompt word, a multimodal large model is used to perform passenger flow anomaly detection on multiple surveillance video frames to obtain anomaly detection results.

[0105] The target prompt words are used as the input of the multimodal large model, and the multimodal large model is used to perform passenger flow anomaly detection on multiple surveillance video frames to obtain the anomaly detection results of the target passenger flow anomaly category, thereby reducing the number of detection video frames.

[0106] In an optional implementation, the above step S301 generates a target prompt word according to the target passenger flow abnormality category, including:

[0107] Generate target prompt words based on target passenger flow abnormality category and preset detection result template;

[0108] In the above step S302, based on the target prompt word, a multimodal large model is used to perform passenger flow anomaly detection on multiple surveillance video frames to obtain anomaly detection results, including:

[0109] According to the target prompt words, a multimodal large model is used to detect passenger flow anomalies on multiple surveillance video frames, and the anomaly detection results of the target passenger flow anomaly category laid out with a preset detection result template are obtained.

[0110] Among them, according to the target passenger flow abnormality category and the preset detection result template, the target prompt word is generated. The target prompt word is used to guide the multimodal large model to judge the target passenger flow abnormality category for each monitoring video frame, and return the abnormal detection result according to the preset detection result template.

[0111] The target prompt words are used as the input of the multimodal large model, and the multimodal large model is used to perform passenger flow anomaly detection on each surveillance video frame to obtain the anomaly detection results of the target passenger flow anomaly category laid out with a preset detection result template.

[0112] For example, if the target passenger flow anomaly categories include crowds and falls, the target prompt words are used to guide the multimodal large model to make the following judgments: whether there are crowds or falls, and the preset detection result template is:

[0113] {'Are there any people gathering':','Are there any people falling':'}

[0114] The multimodal large model is used to detect passenger flow anomalies in each surveillance video frame, and the anomaly detection results of the target passenger flow anomaly category arranged in a preset detection result template can be obtained as follows:

[0115] {"People gathering":"Yes","People falling":"No"}

[0116] The anomaly detection result indicates that there are passenger flow anomalies of the personnel clustering category in the monitored area, but there are no passenger flow anomalies of the personnel iteration.

[0117] In some embodiments, a target Uniform Resource Locator (URL) is configured, and an InternVL2Client instance is created to interact with an Application Programming Interface (API) of a multimodal large model to connect to a backend multimodal large model, query the regional surveillance video according to a video file query request (imagePath), and use an open function to read binary data (video frames) in binary mode to load the video frames of the regional surveillance video, wherein, if the regional surveillance video cannot be queried (i.e., the video path is wrong), an error prompt is pushed to enable the user to check in time, and then the target prompt word and video frame (image_bytes) are used as input parameters, the target prompt word and video frame are serialized into JSON format, and sent to the API of the multimodal large model via a POST request to call the multimodal large model, wherein the Hypertext Transfer Protocol (HTTP) response status code can also be captured to verify whether the request is sent successfully, and then, the multimodal large model is used to perform passenger flow anomaly detection on each surveillance video frame, and a string is returned for parsing to extract the anomaly detection result.

[0118] In the specific implementation, regular expressions or json.loads can be used to parse the string returned by the multimodal large model, extract and convert the text field content (anomaly detection result), verify the integrity and validity of the parsed data (such as checking whether the fields of the returned target passenger flow anomaly category are consistent with expectations), and convert the anomaly detection result format into a standard JSON object. The anomaly detection result can be, for example: {"people gathering":"yes","people falling":"no"}, and the anomaly detection result is returned to the API of the multimodal large model through HTTP response.

[0119] In an optional embodiment, the method may further include:

[0120] According to the date information and time unit, the station passenger flow data is grouped and summarized to generate a passenger flow analysis chart corresponding to the time unit under the date information.

[0121] The station passenger flow data is the passenger flow data of the target station in the time unit under the date information, and the time unit is the time granularity under the date information, for example, hours or minutes.

[0122] Taking the date information as January 1, 2024 and the time unit as hours as an example, the station passenger flow data includes the station passenger flow data of the target station every hour on January 1, 2024.

[0123] The station passenger flow data is grouped and summarized according to the date information and the time unit to obtain a passenger flow analysis chart corresponding to the time unit under the date information. The grouping and summarizing refers to grouping and summarizing the station passenger flow data of the time unit under the date information based on the time unit. The passenger flow analysis chart can be in the form of a line chart or a bar chart, for example, which is not particularly limited in this embodiment.

[0124] In some embodiments, the station passenger flow data is passed to a visualization module, the station passenger flow data is grouped and summarized according to time information, a passenger flow analysis chart is generated using a visualization library, and the chart file is saved or a chart object is returned.

[0125] Take the date information as January 1, 2024, and the time unit as hours as an example. Figure 5 A schematic diagram of the front-end interface provided in the embodiment of the present application, such as Figure 5 As shown, the front-end interface displays a text input box. The user enters the information and time information of the target station through the text input box. The station passenger flow data is grouped and summarized according to the date information and time unit, and the passenger flow analysis chart corresponding to the time unit under the date information is obtained, and the passenger flow analysis chart is displayed on the front-end interface.

[0126] Figure 6 Schematic diagram of the process of the passenger flow warning analysis method provided in the embodiment of the present application Figure 5 ,like Figure 6 As shown, in an optional implementation, the method may further include:

[0127] S501. Load a preset station passenger flow report template.

[0128] S502: Generate a station passenger flow report for the target station using a station passenger flow report template according to the early warning analysis results and the passenger flow analysis chart.

[0129] The preset station passenger flow report template is used to indicate the layout of the early warning analysis results and the passenger flow analysis chart, for example, the layout with the early warning analysis results in front and the passenger flow analysis chart in the back.

[0130] Load the preset station passenger flow report template, fill the early warning analysis results and passenger flow analysis charts into the corresponding positions of the station passenger flow report template according to the layout indicated by the station passenger flow report template, and generate the station passenger flow report of the target station. In this way, the early warning analysis results and passenger flow analysis charts can be integrated into the station passenger flow report, and users can view them intuitively at the same time.

[0131] Among them, in the station passenger flow report, the early warning analysis title can also be displayed at the early warning analysis result and the passenger flow analysis chart title can be displayed at the passenger flow analysis chart.

[0132] In some embodiments, the integrity (whether early warning analysis results and passenger flow analysis charts are missing) and format (such as title format, layout format) of the station passenger flow report can also be checked. After the integrity check and format check are passed, the complete station passenger flow report can be output and provided to users for download or directly pushed to a designated platform for users to view.

[0133] In this embodiment, the interactivity and predictiveness of station passenger flow reports are achieved by intelligently understanding the user's complex query requirements and dynamically generating multi-dimensional analysis and visualization charts and early warning analysis results. This interactive station passenger flow report greatly simplifies the complexity of deployment and integration and can quickly adapt to different business environments and system architectures.

[0134] Figure 7 This is a schematic diagram of the structure of a passenger flow warning and analysis device provided in an embodiment of the present application, which can be integrated into an electronic device.

[0135] like Figure 7 As shown, the device may include:

[0136] The acquisition module 601 is used to acquire the information required for passenger flow warning, which includes: information of the target station, information of the area to be monitored in the target station, target passenger flow abnormality category and time information;

[0137] The acquisition module 601 is further used to acquire the station passenger flow data of the target station and the regional monitoring video of the area to be monitored under the time information according to the information of the target station and the information of the area to be monitored;

[0138] The acquisition module 601 is further used to acquire target passenger flow warning information according to the target passenger flow abnormality category;

[0139] The processing module 602 is used to perform passenger flow anomaly detection on the regional monitoring video according to the target passenger flow anomaly category, and obtain an anomaly detection result of the target passenger flow anomaly category;

[0140] The processing module 602 is also used to generate warning analysis results for the target station using a natural language large model based on the station passenger flow data, target passenger flow warning information and anomaly detection results.

[0141] In an optional implementation, the acquisition module 601 is specifically configured to:

[0142] Get the target station information and time information entered through the text input box;

[0143] In response to the information input operation of the target station, the area selection box and the passenger flow abnormality selection box of the target station are displayed;

[0144] Obtain the information of the area to be monitored and the target passenger flow anomaly category entered through the area selection box and the passenger flow anomaly selection box respectively.

[0145] In an optional implementation, the processing module 602 is specifically configured to:

[0146] Generate target prompt words according to target passenger flow abnormality category;

[0147] According to the target prompt words, a multimodal large model is used to detect passenger flow anomalies in the regional surveillance video and obtain the anomaly detection results.

[0148] In an optional implementation, the processing module 602 is specifically configured to:

[0149] Perform frame extraction processing on the regional surveillance video to obtain multiple surveillance video frames;

[0150] According to the target prompt words, a multimodal large model is used to detect passenger flow anomalies on multiple surveillance video frames to obtain anomaly detection results.

[0151] In an optional implementation, the processing module 602 is specifically configured to:

[0152] Generate target prompt words based on target passenger flow abnormality category and preset detection result template;

[0153] According to the target prompt words, a multimodal large model is used to detect passenger flow anomalies on multiple surveillance video frames, and the anomaly detection results of the target passenger flow anomaly category laid out with a preset detection result template are obtained.

[0154] In an optional implementation, the processing module 602 is specifically configured to:

[0155] Load the prompt word template corresponding to the multimodal large model;

[0156] According to the target passenger flow abnormality category, fill in the prompt word template and generate the target prompt word.

[0157] In an optional implementation, the time information includes: date information and a time unit under the date information, and the processing module 602 is specifically used to:

[0158] According to the date information and time unit, the station passenger flow data is grouped and summarized to generate a passenger flow analysis chart corresponding to the time unit under the date information.

[0159] In an optional implementation, the processing module 602 is specifically configured to:

[0160] Load the preset station passenger flow report template;

[0161] According to the early warning analysis results and passenger flow analysis charts, the station passenger flow report template is used to generate the station passenger flow report of the target station.

[0162] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0163] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 8 As shown, the device may include: a processor 701, a memory 702 and a bus 703. The memory 702 stores machine-readable instructions executable by the processor 701. When the electronic device is running, the processor 701 communicates with the memory 702 through the bus 703, and the processor 701 executes the machine-readable instructions to perform the above method.

[0164] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above method is executed.

[0165] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0166] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0169] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0170] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0171] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A passenger flow early warning analysis method, characterized in that: include: Acquire information required for passenger flow warning, the information required for passenger flow warning includes: information of target station, information of the area to be monitored in the target station, target passenger flow abnormality category and time information; According to the information of the target station and the information of the area to be monitored, obtaining the station passenger flow data of the target station under the time information and the regional monitoring video of the area to be monitored; According to the target passenger flow abnormality category, obtaining target passenger flow warning information; According to the target passenger flow abnormality category, performing passenger flow abnormality detection on the regional surveillance video to obtain an abnormality detection result of the target passenger flow abnormality category; Based on the station passenger flow data, the target passenger flow warning information and the abnormal detection results, a natural language large model is used to generate a warning analysis result for the target station.

2. The method according to claim 1, characterized in that: The information required for obtaining passenger flow warning includes: Acquire the target station information and the time information input through the text input box; In response to the information input operation of the target station, displaying the area selection box and the passenger flow abnormality selection box of the target station; The information of the area to be monitored and the target passenger flow abnormality category inputted through the area selection box and the passenger flow abnormality selection box are obtained respectively.

3. The method according to claim 1, characterized in that The method of performing passenger flow anomaly detection on the regional surveillance video according to the target passenger flow anomaly category to obtain an anomaly detection result of the target passenger flow anomaly category includes: generating a target prompt word according to the target passenger flow abnormality category; According to the target prompt word, a multimodal large model is used to perform passenger flow anomaly detection on the regional surveillance video to obtain the anomaly detection result.

4. The method according to claim 3, characterized in that The method of using a multimodal large model to perform passenger flow anomaly detection on the regional surveillance video according to the target prompt word to obtain the anomaly detection result includes: Performing frame extraction processing on the regional surveillance video to obtain multiple surveillance video frames; According to the target prompt word, the multimodal large model is used to perform passenger flow anomaly detection on the multiple surveillance video frames to obtain the anomaly detection result.

5. The method according to claim 4, characterized in that The step of generating a target prompt word according to the target passenger flow abnormality category includes: Generate the target prompt word according to the target passenger flow abnormality category and a preset detection result template; The method of performing passenger flow anomaly detection on the plurality of surveillance video frames using the multimodal large model according to the target prompt word to obtain the anomaly detection result includes: According to the target prompt word, the multimodal large model is used to perform passenger flow anomaly detection on the multiple surveillance video frames to obtain anomaly detection results of the target passenger flow anomaly category arranged according to the preset detection result template.

6. The method according to claim 3, characterized in that The step of generating a target prompt word according to the target passenger flow abnormality category includes: Loading the prompt word template corresponding to the multimodal large model; According to the target passenger flow abnormality category, the prompt word template is filled to generate the target prompt word.

7. The method according to claim 1, characterized in that The time information includes: date information, and a time unit under the date information, and the method further includes: The station passenger flow data is grouped and summarized according to the date information and the time unit, and a passenger flow analysis chart corresponding to the time unit under the date information is generated.

8. The method according to claim 7, characterized in that The method further comprises: Load the preset station passenger flow report template; According to the early warning analysis result and the passenger flow analysis chart, the station passenger flow report template is used to generate a station passenger flow report for the target station.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the method described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is executed.

Images