Multi-area monitoring system and method based on AI visual model

Abstract

The invention relates to the technical field of risk analysis, and discloses a multi-area monitoring system and method based on an AI vision model.The multi-area monitoring method based on the AI vision model comprises the following steps that area information is collected; generating two-dimensional structure data based on the region information; inputting the two-dimensional structure data into a density model, wherein the density model outputs a result representing the people flow density of each region; inputting the two-dimensional structure data into a management and control model, wherein the management and control model outputs a result representing people flow management and control measures; according to the invention, by analyzing the real-time region information of each region and combining the historical people flow data of the region, the influence of people shielding on the people flow density calculation accuracy is reduced, and the people flow management and control measures are obtained according to the people flow density of each region. The system can provide management and control measures which are good in people flow density reduction effect and low in execution cost, and the safety of all areas is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a multi-region monitoring system and method based on an AI vision model. Background Art

[0002] With the acceleration of urbanization and the change of consumption patterns, shopping malls have become an indispensable part of people's daily lives; however, high passenger flow also brings safety hazards. Crowded areas are prone to accidents, causing casualties and property losses. Deep learning technology is used to integrate video data from different areas to timely detect and warn of congestion; however, in high-density areas, there may be occlusions between people, resulting in inaccurate estimation of crowd density; at the same time, when it is identified that the crowd density in the area exceeds the threshold, the crowd control measures in each area should be adjusted in real time to improve the safety of the shopping mall. Summary of the invention

[0003] The purpose of the present invention is to provide a multi-region monitoring system and method based on AI vision model in order to solve the above problems.

[0004] The present invention provides a multi-region monitoring method based on an AI visual model, comprising the following steps:

[0005] Step 100, collecting regional information, which is collected once at each time sampling point, including exit information, entrance information, restroom information, and functional area information;

[0006] Step 200, generating two-dimensional structure data based on the area information, the two-dimensional structure data includes a data matrix and a relationship matrix, one unit of the data matrix represents one-dimensional structure data of an independent object, the independent object includes an exit, an entrance, a restroom and a functional area, and one unit of the data matrix only contains the area information of the independent object it represents;

[0007] The element in the i-th row and j-th column of the relationship matrix represents the association between the i-th unit of the data matrix and the independent object represented by the j-th unit. If there is an association, the value of this element of the relationship matrix is ​​1, otherwise it is 0;

[0008] There are connections between the interconnected exits, entrances, restrooms, and functional areas;

[0009] The one-dimensional structure data includes n one-dimensional data items ordered by time, and the t-th one-dimensional data item represents the area information collected at the t-th moment;

[0010] Step 300, inputting the two-dimensional structure data into a density model, the density model includes a first intermediate layer, a second intermediate layer, a third intermediate layer, a first feature fusion layer and a first output layer, and the density model outputs a result representing the density of people flow in each area;

[0011] Step 400, input the two-dimensional structure data into the control model, the control model includes a fourth intermediate layer, a fifth intermediate layer, a sixth intermediate layer, a second feature fusion layer and a second output layer, and the control model output represents the result of the crowd control measures.

[0012] Furthermore, the time sampling point refers to the moment at which data is collected at a predetermined time interval, and the default value of the predetermined time interval is 60 seconds.

[0013] Furthermore, the exit information, entrance information, restroom information, and functional area information are input into the density model and control model after feature engineering;

[0014] Exit information, entrance information, restroom information, and functional area information include video and text modalities. For data of various modalities, corresponding feature engineering methods are used to encode and generate features, and then spliced;

[0015] The information of video modality is used as a feature engineering method through CNN algorithm;

[0016] The text modality information is used as a feature engineering method through the Word Embedding algorithm.

[0017] Furthermore, in the density model, the one-dimensional structure data No. 1 is input into the first intermediate layer, the first intermediate layer outputs the first intermediate representation data to the first feature fusion layer, the one-dimensional structure data No. 2 is input into the third intermediate layer, the third intermediate layer outputs the third intermediate representation data to the first feature fusion layer, the first feature fusion layer outputs the first fusion representation data to the second intermediate layer, the second intermediate layer also inputs the relationship matrix, the second intermediate layer outputs the second intermediate representation data to the first output layer, and the first output layer outputs the result representing the pedestrian density of each area.

[0018] Furthermore, the crowd density is encoded into No. 2 one-dimensional structure data; the No. 2 one-dimensional structure data includes B No. 2 data items sorted by time, and the zth No. 2 data item represents the crowd density output by the density model at the zth moment.

[0019] Furthermore, the loss function calculation method of the density model is:

[0020] ;

[0021] in represents the density loss value, represents the total number of density samples, represents the total number of regions of density samples, Indicates The density sample The actual flow density of people in each block, The output of the density model The density sample The density of traffic in a block.

[0022] Furthermore, in the control model, the one-dimensional structure data No. 1 is input into the fourth intermediate layer, the fourth intermediate layer outputs the fourth intermediate representation data to the second feature fusion layer, the one-dimensional structure data No. 2 is input into the fifth intermediate layer, the fifth intermediate layer outputs the fifth intermediate representation data to the second feature fusion layer, the second feature fusion layer outputs the second fusion representation data to the sixth intermediate layer, the sixth intermediate layer also inputs the relationship matrix, the sixth intermediate layer outputs the sixth intermediate representation data to the second output layer, and the second output layer outputs the results of the crowd control measures, including opening or closing exits, opening or closing entrances, adjusting and guiding crowd flow, temporarily isolating certain areas, and initiating emergency evacuation plans.

[0023] Furthermore, the steps of training the control model include:

[0024] Step 401, initializing the parameters of the control model;

[0025] Step 402: Observe the regional information at time e , the strategy executed at time e , regional information at time e+1 , execution strategy Rewards ;

[0026] Step 403, then calculate the policy error:

[0027] ;

[0028] in represents the policy error at time e, represents the discount factor, , Represents the control model input The maximum probability value in the first output vector output when Represents the control model input The first output vector corresponding to the strategy The probability value of

[0029] ;

[0030] in Indicates the total number of regions, Indicates Areas in The density of people flow at any time, Indicates Areas in The density of people flow at any time, For the The weight coefficient of the region, For strategy The execution cost, Score the abortion status. , , Represent the first, second, and third weight coefficients of the reward respectively;

[0031] Step 404: Update the control model. The updated formula is as follows:

[0032] ;

[0033] , represents the step size of deep learning, Indicates delivery update;

[0034] Step 405, iterate steps 102-104 until the control model converges or the number of iterations reaches a set value.

[0035] Further, The calculation method is:

[0036] ;

[0037] in, Indicates the total number of regions, For the The weight coefficient of the region satisfies , Indicates The flow status score of an area is calculated as follows:

[0038] ;

[0039] in Indicates Areas in The density of people flow at any time, Indicates The safe threshold of pedestrian density in an area.

[0040] The present invention provides a multi-region monitoring system based on an AI visual model, which is used to store computer-readable instructions. When the computer-readable instructions are read, the aforementioned multi-region monitoring method based on an AI visual model can be executed.

[0041] The beneficial effects of the present invention are as follows: the present invention reduces the influence of personnel obstruction on the accuracy of crowd density calculation by analyzing the real-time regional information of each area and combining the historical crowd flow data of the area, and obtains crowd control measures according to the crowd density of each area. Compared with only calculating the crowd density, the system can provide control measures with good crowd density reduction effect and low execution cost, thereby improving the safety of each area. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of a multi-region monitoring method based on an AI visual model of the present invention. DETAILED DESCRIPTION

[0043] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the present specification. Various examples may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0044] At least one embodiment of the present invention discloses a risk control assessment method based on multi-dimensional perception and enterprise data quantification, such as Figure 1 As shown, the following steps are included:

[0045] Step 100, collecting regional information, which is collected once at each time sampling point, including exit information, entrance information, restroom information, and functional area information;

[0046] Exit information includes the switch status, location information, video information, exit width, and exit type of the exit area;

[0047] The entrance information includes the switch status, location information, video information, entrance width, and entrance type of the entrance area;

[0048] Restroom information includes the switch status of the restroom area, location information, video information, restroom occupancy status, and restroom type;

[0049] Functional area information includes the area category of the functional area (such as shops, corridors, dining areas, parking lots), switch status, location information, video information, and area of ​​the functional area;

[0050] The time sampling point refers to the moment when data is collected at a predetermined time interval;

[0051] In one embodiment of the present invention, the default value of the predetermined time interval is 60 seconds.

[0052] Step 200, generating two-dimensional structure data based on the area information, the two-dimensional structure data includes a data matrix and a relationship matrix, one unit of the data matrix represents one-dimensional structure data of an independent object, the independent object includes an exit, an entrance, a restroom and a functional area, and one unit of the data matrix only contains the area information of the independent object it represents;

[0053] The element in the i-th row and j-th column of the relationship matrix represents the association between the i-th unit of the data matrix and the independent object represented by the j-th unit. If there is an association, the value of this element of the relationship matrix is ​​1, otherwise it is 0;

[0054] There are connections between the interconnected exits, entrances, restrooms, and functional areas;

[0055] The one-dimensional structure data includes n one-dimensional data items ordered by time, and the t-th one-dimensional data item represents the area information collected at the t-th moment;

[0056] In one embodiment of the present invention, the exit information, entrance information, restroom information, and functional area information are input into the density model and the control model after feature engineering;

[0057] Exit information, entrance information, restroom information, and functional area information include video and text modalities. For data of various modalities, corresponding feature engineering methods are used to encode and generate features, and then spliced;

[0058] The information of video modality is used as a feature engineering method through CNN algorithm;

[0059] The text modality information is used as a feature engineering method through the Word Embedding algorithm.

[0060] Step 300, input the two-dimensional structure data into the density model, the density model includes a first intermediate layer, a second intermediate layer, a third intermediate layer, a first feature fusion layer and a first output layer, wherein the first one-dimensional structure data is input into the first intermediate layer, the first intermediate layer outputs the first intermediate representation data to the first feature fusion layer, the second one-dimensional structure data is input into the third intermediate layer, the third intermediate layer outputs the third intermediate representation data to the first feature fusion layer, the first feature fusion layer outputs the first fusion representation data to the second intermediate layer, the second intermediate layer also inputs the relationship matrix, the second intermediate layer outputs the second intermediate representation data to the first output layer, and the first output layer outputs the result representing the density of people flow in each area;

[0061] In one embodiment of the present invention, the crowd density is encoded as second-dimensional one-dimensional structure data; the second-dimensional one-dimensional structure data includes B second-data items sorted by time, and the zth second-data item represents the crowd density output by the density model at the zth moment.

[0062] In one embodiment of the present invention, the formula of the first intermediate layer is as follows:

[0063]

[0064] in Represents the t-th first intermediate representation data of the v-th unit of the data matrix in the two-dimensional structure data, Represents the t-1th first intermediate representation data of the vth unit of the data matrix in the two-dimensional structure data, Represents the tth data item of the one-dimensional structure data corresponding to the vth unit of the data matrix in the two-dimensional structure data, and are the first and second weight parameters of the first intermediate layer, is the bias parameter of the first intermediate layer, and tanh is the hyperbolic tangent function.

[0065] In one embodiment of the present invention, the formula of the third intermediate layer is as follows:

[0066]

[0067] in Indicates A third intermediate representation data, Indicates A third intermediate representation data, The first one-dimensional structure data The second data item, and are the first and second weight parameters of the third intermediate layer, is the bias parameter of the third intermediate layer, and tanh is the hyperbolic tangent function.

[0068] In one embodiment of the present invention, the formula of the first feature fusion layer is as follows:

[0069]

[0070] in, The first fusion representation data of the vth unit of the data matrix in the two-dimensional structure data, represents the fusion function (concatenation function or summation function), Represents the vth unit of the data matrix in the two-dimensional structure data The first intermediate representation data, Indicates A third intermediate representation data, represents the sum weight matrix of the first feature fusion layer, Represents the summed bias parameter of the first feature fusion layer.

[0071] In one embodiment of the present invention, the formula of the second intermediate layer is as follows:

[0072]

[0073] in The second intermediate representation data representing the vth cell of the data matrix, is the set of cells of the data matrix that are associated with the vth cell of the data matrix, , , and The first fusion representation data of the independent objects represented by the vth and uth cells of the data matrix, respectively, is a learnable scalar parameter, represents the two-dimensional recognition weight parameter of the second intermediate layer, and MLP represents a multi-layer perceptron.

[0074] In one embodiment of the present invention, the formula of the first output layer is as follows:

[0075]

[0076] in represents a flow density vector. The cth component value of the flow density vector represents the flow density of the cth area. The second intermediate representation data representing the vth cell of the data matrix, represents the set of cells of all data matrices, is the weight parameter of the first output layer, is the bias parameter of the first output layer, Represents the sigmoid function.

[0077] In one embodiment of the present invention, the loss function calculation method of the density model is:

[0078]

[0079] in represents the density loss value, represents the total number of density samples, represents the total number of regions of density samples, Indicates The density sample The actual flow density of people in each block, The output of the density model The density sample The density of traffic in a block.

[0080] In one embodiment of the present invention, when the pedestrian flow density in a certain area exceeds its threshold, the second one-dimensional structure data is updated and the control model is started.

[0081] Step 400, input the two-dimensional structure data into the control model, the control model includes a fourth intermediate layer, a fifth intermediate layer, a sixth intermediate layer, a second feature fusion layer and a second output layer, wherein the first one-dimensional structure data is input into the fourth intermediate layer, the fourth intermediate layer outputs fourth intermediate representation data to the second feature fusion layer, the second one-dimensional structure data is input into the fifth intermediate layer, the fifth intermediate layer outputs fifth intermediate representation data to the second feature fusion layer, the second feature fusion layer outputs second fusion representation data to the sixth intermediate layer, the sixth intermediate layer also inputs a relationship matrix, the sixth intermediate layer outputs sixth intermediate representation data to the second output layer, and the second output layer outputs the result of the crowd control measures;

[0082] In one embodiment of the present invention, the formula of the fourth intermediate layer is as follows:

[0083]

[0084]

[0085]

[0086]

[0087] in, , , represents the first, second, and third weight parameters of the fourth intermediate layer, , , represents the first, second, and third bias parameters of the fourth intermediate layer, represents the dot product, , and Respectively represent the first, second, and third intermediate states of the vth unit of the data matrix in the two-dimensional structure data, where Represents the tth data item of the one-dimensional structure data corresponding to the vth unit of the data matrix in the two-dimensional structure data, and Respectively represent the t-th and t-1-th fourth intermediate representation data of the v-th unit of the data matrix in the two-dimensional structure data, n≥t≥1, n represents the total number of the first data items of the first one-dimensional structure data, t=1 , tanh is the hyperbolic tangent function, Represents the sigmoid function.

[0088] In one embodiment of the present invention, the formula of the fifth intermediate layer is as follows:

[0089]

[0090] in Indicates The fifth intermediate representation data, Indicates The fifth intermediate representation data, The first one-dimensional structure data The second data item, and are the first and second weight parameters of the fifth intermediate layer, is the bias parameter of the fifth intermediate layer, and tanh is the hyperbolic tangent function.

[0091] In one embodiment of the present invention, the formula of the second feature fusion layer is as follows:

[0092]

[0093] in, The second fusion representation data of the vth unit of the data matrix in the two-dimensional structure data, represents the fusion function (concatenation function or summation function), Represents the vth unit of the data matrix in the two-dimensional structure data A fourth intermediate representation data, Indicates The fifth intermediate representation data, represents the sum weight matrix of the second feature fusion layer, Represents the summed bias parameter of the second feature fusion layer.

[0094] In one embodiment of the present invention, the formula of the sixth intermediate layer is as follows:

[0095]

[0096]

[0097]

[0098] in The sixth intermediate representation data representing the vth cell of the data matrix, , , and The second fusion representation data representing the independent objects represented by the vth and uth cells of the data matrix, respectively, represents the attention coefficient of the vth and uth units, represents the attention coefficient of the vth and sth units, represents the normalized attention coefficients of the vth and uth units, represents the set of cells of the data matrix that are associated with the vth cell of the data matrix, represents an exponential function with a natural constant as base, represents the trainable attention vector parameter, CONCAT represents the concatenation operation, represents transpose, represents the two-dimensional recognition weight parameter of the sixth intermediate layer, and LeakyReLU is the LeakyReLU activation function.

[0099] In one embodiment of the present invention, the formula of the second output layer is as follows:

[0100]

[0101] in Represents a control vector. The dth component value of the control vector represents the probability value of the dth strategy. The strategy with the largest probability value is selected as the output. The strategy group contains all executable strategies. A strategy includes opening or closing exits, opening or closing entrances, adjusting and guiding the flow of people, temporarily isolating certain areas, and initiating emergency evacuation plans. The sixth intermediate representation data representing the vth cell of the data matrix, represents the set of cells of all data matrices, is the weight parameter of the second output layer, is the bias parameter of the second output layer, Represents the sigmoid function.

[0102] In one embodiment of the present invention, the steps of training the management model include:

[0103] Step 401, initializing the parameters of the control model;

[0104] Step 402: Observe the regional information at time e , the strategy executed at time e , regional information at time e+1 , execution strategy Rewards ;

[0105] Step 403, then calculate the policy error:

[0106]

[0107] in represents the policy error at time e, represents the discount factor, , Represents the control model input The maximum probability value in the first output vector output when Represents the control model input The first output vector corresponding to the strategy The probability value of

[0108]

[0109] in Indicates the total number of regions, Indicates Areas in The density of people flow at any time, Indicates Areas in The density of people flow at any time, For the The weight coefficient of the region, For strategy The execution cost, Score the abortion status. , , Represent the first, second, and third weight coefficients of the reward respectively;

[0110] Step 404: Update the control model. The updated formula is as follows:

[0111]

[0112] , represents the step size of deep learning, Indicates delivery update;

[0113] Step 405, iterate steps 102-104 until the control model converges or the number of iterations reaches a set value. The default value of this value is 1000.

[0114] In one embodiment of the present invention, The calculation method is:

[0115]

[0116] in, Indicates the total number of regions, For the The weight coefficient of the region satisfies , Indicates The flow status score of an area is calculated as follows:

[0117]

[0118] in Indicates Areas in The density of people flow at any time, Indicates The safe threshold of pedestrian density in an area.

[0119] In at least one embodiment of the present invention, a multi-region monitoring system based on an AI visual model is provided, which is used to store computer-readable instructions, and when the computer-readable instructions are read, the aforementioned multi-region monitoring method based on an AI visual model can be executed.

[0120] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation mode. The above-mentioned specific implementation mode is merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make more forms of equivalent embodiments, all of which are within the protection of this embodiment.

Claims

1. A multi-region monitoring method based on AI visual model, characterized in that: include: Step 100, collecting regional information, which is collected once at each time sampling point, including exit information, entrance information, restroom information, and functional area information; Step 200, generating two-dimensional structure data based on the area information, the two-dimensional structure data includes a data matrix and a relationship matrix, one unit of the data matrix represents one-dimensional structure data of an independent object, the independent object includes an exit, an entrance, a restroom and a functional area, and one unit of the data matrix only contains the area information of the independent object it represents; The element in the i-th row and j-th column of the relationship matrix represents the association between the i-th unit of the data matrix and the independent object represented by the j-th unit. If there is an association, the value of this element of the relationship matrix is ​​1, otherwise it is 0; There are connections between the interconnected exits, entrances, restrooms, and functional areas; The one-dimensional structure data includes n number-one data items sorted by time, and the t-th number-one data item represents the area information collected at the t-th moment; Step 300, inputting the two-dimensional structure data into a density model, the density model includes a first intermediate layer, a second intermediate layer, a third intermediate layer, a first feature fusion layer and a first output layer, and the density model outputs a result representing the density of people flow in each area; Step 400, input the two-dimensional structure data into the control model, the control model includes a fourth intermediate layer, a fifth intermediate layer, a sixth intermediate layer, a second feature fusion layer and a second output layer, and the control model output represents the result of the crowd control measures.

2. According to the multi-region monitoring method based on AI visual model in claim 1, it is characterized in that: The time sampling point refers to the moment when data is collected at a predetermined time interval. The default value of the predetermined time interval is 60 seconds.

3. The multi-region monitoring method based on AI visual model according to claim 1 is characterized in that: After feature engineering, the exit information, entrance information, restroom information, and functional area information are input into the density model and control model; Exit information, entrance information, restroom information, and functional area information include video and text modalities. For data of various modalities, corresponding feature engineering methods are used to encode and generate features, and then spliced; The information of video modality is used as a feature engineering method through CNN algorithm; The text modality information is used as a feature engineering method through the Word Embedding algorithm.

4. The multi-region monitoring method based on AI visual model according to claim 1 is characterized in that: In the density model, the one-dimensional structure data No. 1 is input into the first intermediate layer, the first intermediate layer outputs the first intermediate representation data to the first feature fusion layer, the one-dimensional structure data No. 2 is input into the third intermediate layer, the third intermediate layer outputs the third intermediate representation data to the first feature fusion layer, the first feature fusion layer outputs the first fusion representation data to the second intermediate layer, the second intermediate layer also inputs the relationship matrix, the second intermediate layer outputs the second intermediate representation data to the first output layer, and the first output layer outputs the result representing the pedestrian density of each area.

5. The multi-region monitoring method based on AI visual model according to claim 4 is characterized in that: The crowd density is encoded into No. 2 one-dimensional structure data; the No. 2 one-dimensional structure data includes B No. 2 data items sorted by time, and the zth No. 2 data item represents the crowd density output by the density model at the zth moment.

6. The multi-region monitoring method based on AI visual model according to claim 4 is characterized in that: The loss function calculation method of the density model is: ； in represents the density loss value, represents the total number of density samples, represents the total number of regions of density samples, Indicates The density sample The actual flow density of people in each block, The output of the density model The density sample The density of traffic in a block.

7. The multi-region monitoring method based on AI visual model according to claim 1, characterized in that: In the control model, the one-dimensional structure data No. 1 is input into the fourth intermediate layer, and the fourth intermediate layer outputs the fourth intermediate representation data to the second feature fusion layer. The one-dimensional structure data No. 2 is input into the fifth intermediate layer, and the fifth intermediate layer outputs the fifth intermediate representation data to the second feature fusion layer. The second feature fusion layer outputs the second fusion representation data to the sixth intermediate layer. The sixth intermediate layer also inputs the relationship matrix, and the sixth intermediate layer outputs the sixth intermediate representation data to the second output layer. The output of the second output layer represents the results of the crowd control measures, including opening or closing exits, opening or closing entrances, adjusting and guiding crowd flow, temporarily isolating certain areas, and initiating emergency evacuation plans.

8. The multi-region monitoring method based on AI visual model according to claim 1, characterized in that: The steps of training the control model include: Step 401, initializing the parameters of the control model; Step 402: Observe the regional information at time e , the strategy executed at time e , regional information at time e+1 , execution strategy Rewards ; Step 403, then calculate the policy error: ； in represents the policy error at time e, represents the discount factor, , Represents the control model input The maximum probability value in the first output vector output when Represents the control model input The first output vector corresponding to the strategy The probability value of ； in Indicates the total number of regions, Indicates Areas in The density of people flow at any time, Indicates Areas in The density of people flow at any time, For the The weight coefficient of the region, For strategy The execution cost, Score the abortion status. , , Represent the first, second, and third weight coefficients of the reward respectively; Step 404: Update the control model. The updated formula is as follows: ； , represents the step size of deep learning, Indicates delivery update; Step 405, iterate steps 102-104 until the control model converges or the number of iterations reaches a set value.

9. The multi-region monitoring method based on AI visual model according to claim 8, characterized in that: The calculation method is: ； in, Indicates the total number of regions, For the The weight coefficient of the region satisfies , Indicates The flow status score of an area is calculated as follows: ； in Indicates Areas in The density of people flow at any time, Indicates The safe threshold of pedestrian density in an area.

10. A multi-region monitoring system based on AI visual model, characterized in that: It is used to store computer-readable instructions, which, when read, can execute a multi-region monitoring method based on an AI visual model as described in any one of claims 1-9.

Images