A method, device, storage medium, system and robot for target detection and analysis based on multiple cameras

Abstract

The present invention is applicable to the field of target detection technology, and provides a target detection analysis method, device, storage medium, system and robot based on multiple cameras. The method includes: obtaining a number of videos to be detected collected by a number of terminal devices, and extracting images to be detected from the videos to be detected, and finally obtaining a work rule model of the target personnel. The present application obtains video data from different cameras and integrates and analyzes them to establish a residence time model of the target personnel in each area, so that the analysis of the personnel's spatiotemporal motion model is more accurate; and it can combine the spatial layout of the monitoring scene, the distribution of work tasks and the regional characteristics to establish a spatiotemporal relationship model, taking into account the frequency and duration of the target personnel's stay in different areas, improving the accuracy of the inference of the personnel's regional residence time, and significantly improving the existing target acquisition analysis model The ability to analyze the movement of personnel.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a target detection and analysis method, device, storage medium, system and robot based on multiple cameras. Background Art

[0002] The core of security monitoring is to capture video images through cameras, and to monitor and store them in real time through video surveillance systems. These systems can include a single camera or multiple cameras, covering different areas and angles, and are used to monitor scenes such as buildings, public places, and traffic intersections.

[0003] The core of security technology lies in motion detection and intelligent analysis algorithms. Motion detection technology can identify moving objects in video surveillance systems and issue alarms or trigger other security response measures when motion is detected. Intelligent analysis algorithms can analyze surveillance videos in real time to identify specific targets, behaviors or events. For example, face recognition technology can be used to identify people appearing in surveillance images, and license plate recognition technology can be used to identify vehicle license plate information.

[0004] In the current security field, especially in the application environment of monitoring systems, the need to monitor the time that personnel stay in a specific area is becoming increasingly prominent. However, traditional monitoring technology faces a series of challenges. First, existing monitoring systems usually can only rely on a single camera to monitor a specific area, and cannot meet the needs of simultaneous monitoring of multiple areas. Secondly, traditional face recognition technology has low recognition accuracy in complex environments, and its ability to track personnel trajectories is limited, and its data analysis capabilities are relatively limited. Summary of the invention

[0005] The purpose of the embodiment of the present application is to provide a target detection and analysis method based on multiple cameras, aiming to solve the problem that the existing monitoring system can usually only rely on a single camera to monitor a specific area and cannot meet the needs of simultaneous monitoring of multiple areas. In addition, the recognition accuracy of traditional face recognition technology is low in complex environments, and its ability to track people's trajectories is limited, and its data analysis ability is relatively limited.

[0006] The embodiment of the present application is implemented by providing a target detection and analysis method based on multiple cameras, the method comprising:

[0007] Acquire a number of videos to be detected collected by a number of terminal devices;

[0008] Extracting an image to be detected and adjacent frame images before and after the image to be detected from the video to be detected, and obtaining a motion vector of each pixel in the image to be detected compared with the adjacent frame images before and after the image to be detected;

[0009] Acquire all the portraits to be tested in all the videos to be tested, perform the same target matching on the portraits to be tested based on the facial similarity feature, the time feature and the spatial feature, and cluster the portraits to be tested representing the same person to obtain several target persons and a set of target portraits associated with the target persons, wherein the time feature is used to represent the displacement relationship of the target person over time, and the spatial feature is used to represent the spatial displacement relationship of the portrait persons;

[0010] Based on the motion vector and the target portrait set, obtaining a moving trajectory of the target person associated with the target portrait set;

[0011] Based on the target person and his / her movement trajectory and residence time in the area to be detected, a work rule matrix of the target person is obtained; based on the work rule matrix and the work task characteristics of the target person in the area to be detected, a work rule model of the target person is obtained.

[0012] Preferably, the method for matching the same target based on face similarity features is:

[0013] Acquire image data from the video to be detected, and perform face detection on the image data to obtain a face image;

[0014] Extracting features from the face image to obtain a feature vector;

[0015] Combining the feature vectors corresponding to all the facial images obtained in the video to be detected to obtain a vector library;

[0016] Perform similarity matching calculation on the feature vector corresponding to the face image of the target person and the feature vector in the vector library to obtain a similarity value;

[0017] The portraits to be tested corresponding to a number of feature vectors whose similarity values ​​are higher than a similarity threshold are set as a target portrait set of the target person;

[0018] The method of extracting features from the face image to obtain a feature vector is:

[0019] Performing face cropping and key feature point detection on the face image;

[0020] Aligning the key feature points based on the detection result of the key feature point detection, and masking the area outside the face in the face image to obtain a preprocessed image;

[0021] A deep learning model is used to extract features from the preprocessed image to obtain a feature vector.

[0022] Preferably, the method for obtaining the motion vector of a pixel is:

[0023] Assume that at time t, the position of a pixel is (x, y), and at time t+1, its position is (x′, y′), then the motion vector (u, v) is calculated as:

[0024]

[0025] Among them, (u, v) represents the t The motion vector of the pixel point (x, y) in , min means to go to the minimum value.

[0026] Preferably, the method for obtaining the work rule matrix of the target personnel is:

[0027] Divide the area to be detected into several sub-areas, and perform three-dimensional modeling on the sub-areas to obtain the spatial position coordinates of each sub-area;

[0028] Obtain the target person's residence time in each sub-area and the movement sequence of entering each sub-area;

[0029] The work rule matrix includes the target personnel's appearance frequency matrix F in each sub-area, the transition probability matrix P between work tasks, and the work time matrix W in each sub-area;

[0030] Based on the residence time and the movement sequence, the occurrence frequency matrix F, the transition probability matrix P and the working duration matrix W are obtained.

[0031] Preferably, the working law model is obtained in the following manner:

[0032] Obtain the work task characteristics W of the target personnel in each sub-area i , as well as the distance matrix D and characteristic matrix S that characterize the relative positions between each sub-area, to obtain the spatiotemporal relationship function of personnel movement;

[0033] Based on the time-space relationship function of the target personnel, the work rule model of the target personnel is obtained.

[0034] Preferably, the method further comprises:

[0035] Acquiring dynamic weight parameters, wherein the dynamic weight parameters at least include a stay frequency calculation or a duration calculation;

[0036] Based on the dynamic weight parameter, the weight of the working rule matrix is ​​dynamically adjusted.

[0037] Another object of an embodiment of the present application is to provide a multi-camera based target detection and analysis device, the multi-camera based target detection and analysis device comprising:

[0038] The video module to be detected is used to obtain a number of videos to be detected collected by a number of terminal devices;

[0039] A motion vector acquisition module is used to extract the image to be detected and the adjacent frame images before and after the image to be detected from the video to be detected, and obtain the motion vector of each pixel in the image to be detected compared with the adjacent frame images before and after it;

[0040] The same target matching module is used to obtain all the portraits to be tested in all the videos to be tested, perform same target matching on the portraits to be tested based on facial similarity features, time features and spatial features, and cluster the portraits to be tested that represent the same person to obtain several target persons and a set of target portraits associated with the target persons. The time features are used to represent the displacement relationship of the target persons over time, and the spatial features are used to represent the spatial displacement relationship of the portrait persons.

[0041] A movement trajectory acquisition module, used for obtaining a movement trajectory of a target person associated with the target portrait set based on the motion vector and the target portrait set;

[0042] The work rule model acquisition module is used to obtain the work rule matrix of the target person based on the target person and his / her movement trajectory and residence time in the area to be detected; and to obtain the work rule model of the target person based on the work rule matrix and the work task characteristics of the target person in the area to be detected.

[0043] Another object of an embodiment of the present application is to provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the target detection and analysis method based on multiple cameras as described above.

[0044] Another object of an embodiment of the present application is to provide a target detection and analysis system based on multiple cameras, wherein when the system is running, the steps of the target detection and analysis method based on multiple cameras as described above are executed.

[0045] Another object of an embodiment of the present application is to provide a robot, on which are provided a plurality of monitoring cameras, and the monitoring cameras perform area monitoring by executing the target detection and analysis method based on multiple cameras as described above.

[0046] The target detection and analysis method based on multiple cameras provided in the embodiment of the present application is beneficial in that: it can obtain video data from different cameras and integrate and analyze them, record the time intervals of the appearance of people in different areas, and establish a residence time model of the target people in each area, so that the analysis of the spatiotemporal motion model of the people is more accurate; and it can combine the spatial layout of the monitoring scene, the distribution of work tasks and the regional characteristics to establish a spatiotemporal relationship model, taking into account the frequency and duration of the target people's stay in different areas, thereby improving the accuracy of the inference of the regional residence time of people, and significantly improving the analysis ability of the existing target acquisition analysis model for the movement of people. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 An application environment diagram of a multi-camera based target detection and analysis method provided in an embodiment of the present application;

[0048] Figure 2 A flowchart of a multi-camera target detection and analysis method provided in an embodiment of the present application;

[0049] Figure 3 A method diagram of facial similarity feature matching provided in an embodiment of the present application;

[0050] Figure 4 A module diagram of a multi-camera target detection and analysis device provided in an embodiment of the present application;

[0051] Figure 5 FIG. 4 is a block diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script without departing from the scope of this application.

[0054] Figure 1 The application environment diagram of the target detection and analysis method based on multiple cameras provided in the embodiment of the present application is as follows: Figure 1 As shown, in this application environment, a video acquisition terminal 110 and a computer device 120 are included.

[0055] The computer device 120 may be an independent physical server or terminal, a desktop computer, or a server cluster composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN.

[0056] The video acquisition terminal 110 can be various cameras, smart phones, tablet computers, laptop computers, etc., but is not limited thereto. The video acquisition terminal 110 and the computer device 120 can be connected via a wired or wireless network, and this application does not limit this.

[0057] like Figure 2 As shown, in one embodiment, a target detection and analysis method based on multiple cameras is proposed. This embodiment mainly uses this method to be applied to the above Figure 1 The computer device 120 in the embodiment can also be applied to the video acquisition terminal 110 including a data processing unit. A method for target detection and analysis based on multiple cameras can specifically include the following steps:

[0058] Step S10, obtaining a plurality of videos to be detected collected by a plurality of terminal devices;

[0059] Step S20, extracting the image to be detected and the adjacent frame images before and after the image to be detected from the video to be detected, and obtaining the motion vector of each pixel in the image to be detected compared with the adjacent frame images before and after the image to be detected;

[0060] Step S30, obtaining all the portraits to be tested in all the videos to be tested, performing the same target matching on the portraits to be tested based on the facial similarity feature, the time feature and the spatial feature, and clustering the portraits to be tested representing the same person to obtain a number of target persons and a set of target portraits associated with the target persons, wherein the time feature is used to represent the displacement relationship of the target person over time, and the spatial feature is used to represent the spatial displacement relationship of the portrait persons;

[0061] Step S40, based on the motion vector and the target portrait set, obtaining a movement trajectory of the target person associated with the target portrait set;

[0062] Step S50, based on the target person and his / her movement trajectory and residence time in the area to be detected, obtain the work rule matrix of the target person; based on the work rule matrix and the work task characteristics of the target person in the area to be detected, obtain the work rule model of the target person.

[0063] In an embodiment of the present application, several videos to be detected can be acquired by different acquisition terminals and processed by the same processing equipment. A large number of images to be detected can be obtained from each segment of images to be detected. The motion vector of the pixel can be obtained by correlating the pixels in the image with the frames before and after it through an algorithm. The motion vector of the pixel representing the portrait can be analyzed to obtain the motion vector of the portrait. The image recognition algorithm can be used to obtain all the portraits to be detected in all the videos to be detected, and the portraits to be detected can be matched with the same target based on the three dimensions of face similarity features, time features, and spatial features.

[0064] In a preferred embodiment, the method for matching the same target based on time features may be:

[0065] Consider the appearance of people at different time points, such as the order of appearance and duration of stay. Suppose the appearance time of the person captured by the first camera is t 1 , the person captured by the second camera appears at time t 2 , then the temporal feature similarity can be measured using the absolute time difference:

[0066] Δt=|t 2 -t 1 |

[0067] The same target matching method based on spatial features can be:

[0068] Consider the position information of the person in space, such as the position coordinates of the person. Let the position coordinates of the person captured by the first camera be represented as P 1 (x 1 ,y 1 ), the position coordinates of the person captured by the second camera are expressed as P 2 (x 2 ,y 2 ), the spatial feature similarity can be measured using the Euclidean distance:

[0069]

[0070] After obtaining the time intervals, these time interval information can be used for personnel matching. Usually, the matching strategy is based on a certain threshold to determine whether two time intervals belong to the same person. For example, a time threshold is set. If the difference between two time intervals is less than the threshold, the two time intervals are considered to belong to the same person. In this way, the detailed movement trajectory of the personnel in the entire area can be obtained, and their stay time in each sub-area can be further inferred, providing detailed data for time series analysis. The same applies to distance features.

[0071] In an embodiment of the present application, a method for analyzing spatial features and temporal features is provided. Those skilled in the art may appreciate that other methods for analyzing temporal features and spatial features of a target person may also be used to achieve continuous positioning of the target person on the timeline and continuous positioning of the spatial position to determine whether the target in the image is the same person.

[0072] The above embodiment provides a method for acquiring time features and space features. Those skilled in the art will appreciate that other algorithms or combinations may be used to acquire the above features.

[0073] In an embodiment of the present application, the respective judgment results obtained by matching the portraits to be tested with the same target based on facial similarity features, time features, and spatial features can be combined by taking the intersection, or can be superimposed by taking the union, or the results can be comprehensively judged by setting weights among the three. There is no restriction on the specific method of the final judgment result here. All the portraits to be tested corresponding to the judgment results of the comprehensive three can be regarded as a set that can represent the same person to form a target portrait set.

[0074] In one embodiment, for videos shot from different viewpoints and different cameras, the perspective of a single camera may not cover all areas, or the perspectives of different cameras may overlap. In order to associate these perspectives together, we need to apply a target matching algorithm. In addition to matching appearance similarity based on facial feature vectors, the target matching algorithm here also refers to spatial and temporal features. By combining the target detection and tracking algorithm, the position of a person can be detected and tracked in a continuous video sequence, and the same person in images captured by different cameras can be associated through a matching algorithm.

[0075] In an embodiment of the present application, the target person in the target portrait set can be analyzed using motion vectors to obtain the movement trajectory of the target person associated with the target portrait set. Then, based on the target person and his / her movement trajectory and residence time in the monitoring area formed by a plurality of cameras, the work rule matrix of the target person is obtained; based on the work rule matrix and the work task characteristics of the target person in the area to be detected, the work rule model of the target person is obtained. The work task characteristics can be work type, work content, importance of work content, etc.

[0076] In an embodiment of the present application, by acquiring video data from different cameras and integrating and analyzing them, the time intervals of the appearance of people in different areas are recorded, and a residence time model of the target people in each area is established, making the analysis of the spatiotemporal motion model of the people more accurate; and it is able to combine the spatial layout of the monitoring scene, the distribution of work tasks, and the regional characteristics to establish a spatiotemporal relationship model, taking into account the frequency and duration of the target people's stay in different areas, thereby improving the accuracy of the inference of the people's regional residence time, and significantly improving the existing target acquisition analysis model's ability to analyze the movement of people.

[0077] In a preferred embodiment of the present application, time interval calculation is one of the basic methods for obtaining various types of residence time. The method for obtaining the time interval can be:

[0078] By analyzing the timestamps of people appearing in different areas, we can calculate their stay time in each area. This process involves recording timestamps, calculating time intervals, and inferring stay time.

[0079] Timestamp recording: The cameras in each area capture the timestamps of people appearing and record the time of each appearance.

[0080] Time interval calculation: Sort the timestamp sequence of each region and calculate the time interval between adjacent timestamps, that is, the residence time.

[0081] Stay time inference: Infer the stay time of people in each area based on the calculated time interval.

[0082] It can be assumed that in the i-th area, the timestamp sequence of the person appearing is T i ={t 1 , t 2 ,…,t n}, where t j represents the timestamp of the jth occurrence. Then the time a person stays in the area can be calculated by the difference between adjacent timestamps:

[0083]

[0084] In a preferred embodiment of the present application, the method further includes: using time series analysis technology to construct time series data based on the residence time of personnel at different locations to identify common residence patterns and abnormal behaviors.

[0085] First, the moving average technique can be used to smooth the residence time series data. By calculating the average of the residence time in the sliding window, the smoothed residence time series is obtained. Moving average helps to smooth the time series, reduce the impact of noise, and make common residence patterns more prominent.

[0086] Assume there is a stay time series S = {s 1 ,s 2 , ..., s m}, where s k represents the residence time at position k. Moving average can be used to smooth the time series. Assuming the window size is k, the residence time series in the window is S w ={s w-k+1 ,s w-k+2 , ..., s w}, the moving average is:

[0087]

[0088] Then, cluster analysis methods can be used to cluster the smoothed residence time series data. The K-means clustering algorithm is used to determine the cluster centers, thereby identifying different residence patterns. This helps to divide the residence time series data into different categories to reveal the characteristics and changing trends of different residence patterns.

[0089] Finally, anomaly detection methods can be used to detect outliers in the dwell time series using statistical methods such as Z-score or machine learning algorithms such as isolation forests to identify possible abnormal behaviors. These outliers may reflect dwell patterns or unusual activities that are inconsistent with regular behavior and deserve further investigation and analysis.

[0090] Through the above time series analysis method, we can better understand the patterns and changes of people's stay behavior, thereby providing more effective decision-making support for areas such as security management and crowd control.

[0091] In a preferred embodiment of the present application, Figure 3 As shown in FIG. 1 , a method for matching the same target based on face similarity features is given, which can be specifically:

[0092] Acquire image data from the video to be detected, and perform face detection on the image data to obtain a face image;

[0093] Extracting features from the face image to obtain a feature vector;

[0094] Combining the feature vectors corresponding to all the facial images obtained in the video to be detected to obtain a vector library;

[0095] Perform similarity matching calculation on the feature vector corresponding to the face image of the target person and the feature vector in the vector library to obtain a similarity value;

[0096] The portraits to be tested corresponding to several feature vectors whose similarity values ​​are higher than the similarity threshold are set as the target portrait set of the target person.

[0097] In the embodiment of the present application, a face detection and recognition model based on deep learning can be deployed on fixed-point cameras in different areas. The camera can capture images in the area and input these images into the face recognition system.

[0098] In a preferred embodiment of the present application, the method of extracting features from the face image to obtain a feature vector is:

[0099] Perform face cropping and key feature point detection on the face image; align the key feature points based on the detection results of the key feature point detection, and mask the area outside the face in the face image to obtain a preprocessed image; use a deep learning model to extract features from the preprocessed image to obtain a feature vector.

[0100] In the embodiments of the present application, Figure 3 As shown, the algorithm model can be used to detect faces in images first, and then perform face cropping and key point detection and alignment. The key points can refer to the five main key points of the face, namely the two eyes, the tip of the nose and the two corners of the mouth. By cropping and aligning, the algorithm can clearly obtain portrait features in harsh environments, when facial expressions, lighting, and postures change, and even when the face is occluded or blurred. The face recognition algorithm can use a comprehensive supervised learning objective and can also include an innovative contrast training component for learning more discriminative facial features.

[0101] Then, the algorithm model is used to extract facial features and to match and retrieve relevant sample information from the facial feature library. For example, each input facial image is encoded into a 128-dimensional vector. This vector can be regarded as a digital fingerprint of the face, that is, a facial feature vector. Then, feature comparison is performed to compare the two facial feature vectors and calculate the distance between them. Cosine similarity and other methods can be used for comparison. Assume that the two feature vectors are v 1 and v 2 , then the cosine similarity calculation formula is:

[0102]

[0103] where · represents the dot product of vectors, |·| 2 Represents the L2 norm of a vector.

[0104] Finally, according to the actual application, a distance threshold is set. If the calculated distance is less than the threshold, it is considered to be the same person; conversely, if it is greater than the threshold, it is considered to be different people. The advantage of facial feature comparison based on this solution is that the extracted facial features have good distinguishability and robustness. It can well cope with changes in factors such as lighting, expression, and posture, and is suitable for many practical face recognition application scenarios. The processing and recognition accuracy performed by this solution is higher.

[0105] In a preferred embodiment of the present application, the motion vector of each pixel in the image can be calculated for each pair of adjacent frames by using, for example, an optical flow method, and these motion vectors are applied to the trajectory estimation of the person, that is, the motion of the object in the image is estimated by minimizing the optical flow field error. The method for obtaining the motion vector of the pixel can be:

[0106] Assume that at time t, the position of a pixel is (x, y), and at time t+1, its position is (x′, y′), then the motion vector (u, v) is calculated as:

[0107]

[0108] Among them, (u, v) represents the t The motion vector of the pixel point (x, y) in , min means to go to the minimum value.

[0109] In a preferred embodiment of the present application, the method for obtaining the work rule matrix of the target personnel may be:

[0110] The area to be detected is divided into several sub-areas, and the sub-areas are three-dimensionally modeled to obtain the spatial position coordinates of each sub-area; the target person's residence time in each sub-area and the movement sequence of entering each sub-area are obtained; the work rule matrix includes the target person's appearance frequency matrix in each sub-area, the transition probability matrix between work tasks and the work duration matrix in each sub-area; based on the residence time and the movement sequence, the appearance frequency matrix, the transition probability matrix and the work duration matrix are obtained.

[0111] In this embodiment, a specific method for obtaining the working rule matrix is ​​provided:

[0112] First, the construction site can be divided into different areas or work stations, recorded as sub-areas, according to the actual situation and needs of the construction site. These areas can represent different construction units, functional areas or work areas. Then, the spatial relationship between the areas is established, and the position and relative position relationship of different areas can be represented by a two-dimensional Cartesian coordinate system.

[0113] Suppose there is a construction site divided into three areas A, B, and C. Use a two-dimensional Cartesian coordinate system to represent the boundaries of different areas. Suppose the boundary of area A consists of the following four vertices: (0,0), (10,0), (10,5), (0,5). These four vertices are connected in clockwise order to form the boundary of area A. Suppose the boundary of area B consists of the following five vertices: (10,0), (20,0), (20,10), (15,15), (10,10). These five vertices are connected in clockwise order to form the boundary of area B. Suppose the boundary of area C consists of the following four vertices: (0,5), (10,5), (10,10), (0,10). These four vertices are connected in clockwise order to form the boundary of area C.

[0114] In this way, by defining the boundary polygon of each area, different sub-areas are successfully distinguished in the two-dimensional Cartesian coordinate system, and the spatial relationship between the areas is established.

[0115] Then, work pattern modeling is carried out to analyze and model the work patterns and behavioral characteristics of personnel at the construction site to obtain a work pattern matrix. This includes research on areas frequently visited by personnel, the order of work tasks, and the distribution of working hours, and relevant information is obtained through historical data analysis or expert experience.

[0116] For each worker, there may be some specific areas where they frequently appear when working on the construction site. These frequently visited areas can be determined by analyzing historical data or observing the behavior of workers. Assuming there are M workers and N areas, an M×N matrix F can be used to represent the frequency of each worker's appearance in each area. Among them, F ij Represents the number of times the i-th worker appears in the j-th area.

[0117] The order of work tasks may be affected by many factors, including project progress, worker skills, etc. The order model of work tasks can be determined by analyzing historical data or discussing with professionals. A simple method is to use a probability model to represent the transition probability between different tasks. Assuming there are K types of work tasks, a K×K transition matrix P can be used to represent the transition probability between tasks. Among them, P ij represents the probability of transferring from task i to task j.

[0118] Working time distribution can describe the distribution of workers’ working hours and working time periods at the construction site. Relevant information can be obtained by statistical historical data or observing workers’ working habits. Assuming there are M workers and T time periods, an M×T matrix W can be used to represent the working hours of each worker in different time periods. Among them, W ijIt represents the working hours of the i-th worker in the j-th time period.

[0119] Suppose there is a construction site with 5 workers and 4 areas. Through historical data analysis, the following information is obtained: the worker's appearance frequency matrix F in each area, the transition probability matrix P between work tasks, and the worker's working time matrix W in different time periods are as follows:

[0120]

[0121] Take the first worker as an example. The above matrix shows that he appeared in the first area 10 times, the second area 5 times, the third area 15 times, and the fourth area 8 times, so the corresponding rows are [10, 5, 15, 8]; secondly, after completing the first task, the probability of transferring to the first task is 0.2, the probability of transferring to the second task is 0.3, the probability of transferring to the third task is 0.1, and the probability of transferring to the fourth task is 0.4, so the corresponding rows are [0.2, 0.3, 0.1, 0.4]; finally, the first worker worked 2 hours in the first time period, 3 hours in the second time period, 4 hours in the third time period, and 5 hours in the fourth time period, so the corresponding rows are [2, 3, 4, 5].

[0122] In a preferred embodiment of the present application, the working law model is obtained in the following manner:

[0123] Obtain the work task characteristics W of the target personnel in each sub-area i , as well as the distance matrix D and characteristic matrix S that characterize the relative positions between each sub-area, to obtain the spatiotemporal relationship function of personnel movement; based on the spatiotemporal relationship function of the target personnel, the work law model of the target personnel is obtained.

[0124] In the embodiment of the present application, the work task characteristic W iThe type of work task in the area can be automatically obtained by using image recognition models, or the task type corresponding to each sub-area can be directly edited or input. By combining spatial layout and work rules when establishing a spatiotemporal relationship model, the relationship between the time personnel stay in different areas and work tasks, the distance between areas, and the characteristics of the areas can be considered, and then the work rule model of the target personnel can be obtained. For the characteristic matrix S, specifically, assuming that there is a construction site with 3 areas, S is a 3x3 matrix. Each column represents a source area, and each row represents a target area. Each element S[i][j] in S represents the characteristics that need to be considered when going from area i to area j. For example, if there are hazardous substances in the destination area, we can set the value of S[i][j] to be higher; on the contrary, if the destination area is safer, the value of S[i][j] is relatively low. In this way, the difficulty or priority of moving from one area to another can be evaluated based on the S matrix. In addition to characterizing the hazardous characteristics, the characteristic matrix S can also be set specifically for specific purposes based on the following aspects: Classification of use areas: For example, an area may be designed as an area dedicated to a specific task, such as a storage area, loading area, mechanical operation area, office area, etc. Classification of physical conditions: such as ambient temperature, humidity, noise level, lighting conditions, ventilation conditions, etc. These factors may directly affect the work efficiency and safety of workers. Classification of the number and type of facilities / equipment: For example, how many machines may be in an area, or how many types of materials are stored. Classification of area capacity: It can be the number of people that the area can accommodate, or the number of tasks that the area can perform. This parameter can help us understand the carrying capacity of the area. Classification of the priority or importance of the area: This can be an important factor when completing a task or an emergency occurs. Some areas may be more important than others.

[0125] The above information is recorded in the characteristic matrix S according to a specific coding method and is represented by numerical values ​​or levels.

[0126] It can be assumed that there are N areas on the construction site, and the time a person stays in the i-th area is T i , the work task is W i , the distance matrix between regions is D, the characteristic matrix is ​​S, and f is the spatiotemporal relationship function, which represents the relationship between the residence time and the work task, regional distance and characteristics. Then:

[0127] T i =f(W i , D ij , S i )

[0128] Based on the above functions, the work rule model of the target personnel can be obtained.

[0129] In a preferred embodiment of the present application, the method further comprises:

[0130] A dynamic weight parameter is obtained, wherein the dynamic weight parameter at least includes a dwell frequency calculation or a duration calculation; and based on the dynamic weight parameter, the weight of the working rule matrix is ​​dynamically adjusted.

[0131] In an embodiment of the present application, in order to reflect the more specific working status of target personnel such as construction personnel, such as the activity level in rest areas and passage areas, since the index of the activity level in these areas may not be very important, specific weights can be applied to such areas so that they do not affect the judgment of the working status of the target personnel in the entire work model. Weights can also be applied to key areas to improve the weight scores of key sub-areas. Dynamic weight calculation is to more accurately reflect the activities of personnel in different work areas, and to improve the accuracy of inference by dynamically adjusting the weight of the residence time.

[0132] In one embodiment, the method for obtaining the dynamic weight is:

[0133] Calculation of stay frequency: Calculate the frequency of stay of personnel in each area, that is, the number of times they appear in the area per unit time.

[0134] Duration calculation: Calculate the average length of time a person spends in each area, which is the total duration of each stay divided by the number of stays.

[0135] Dynamic weight adjustment: Calculate the weight of each area based on the frequency and duration of stay to reflect the degree of activity of people in different areas. Weighted average or other dynamic adjustment methods can be used.

[0136] It can be assumed that the frequency of people staying in the i-th area is F i The average residence time is t i , then the weight ω of the region can be calculated i :

[0137] ω i =α·F i +(1-α)·t i

[0138] Among them, α is the adjustment parameter of the dynamic weight, which can be adjusted according to the actual situation.

[0139] In the embodiment of the present application, a dynamic weight calculation method is introduced to dynamically adjust the weight of the stay time according to the frequency and duration of the stay of the personnel in different areas. This method can flexibly adjust the weight according to the actual situation, better reflect the activities of the personnel in different areas, and improve the accuracy of inference.

[0140] By recording the timestamps of people appearing in different areas and calculating the time intervals, a model of people's residence time in each area was established. Combining the spatial layout of the construction site, the distribution of work tasks and regional characteristics, a spatiotemporal relationship model was established. The frequency and duration of people's stay in different areas were considered, and the weight of the stay time was dynamically adjusted, which improved the accuracy of the inference of people's regional stay time and brought the following advantages:

[0141] (1) Improve construction site management efficiency: By accurately inferring the length of time that various types of personnel stay in different task-type areas, construction managers can better understand the activities on the construction site, allocate resources in a timely manner, and optimize work processes, thereby improving management efficiency.

[0142] (2) Optimize the allocation of construction resources: By accurately inferring the length of stay of each person, the use of personnel and equipment can be arranged more reasonably, avoiding idleness and waste of resources and improving the utilization rate of construction resources.

[0143] (3) Improve construction safety: By real-time monitoring of personnel activities at the construction site, potential safety hazards can be discovered and resolved in a timely manner, reducing the risk of accidents and improving construction safety.

[0144] (4) Improve construction quality: By accurately inferring the residence time, the construction progress and quality can be better monitored, problems in the construction process can be discovered in time and adjustments can be made, thereby improving the construction quality.

[0145] (5) Reduce construction costs: By optimizing resource utilization and improving construction efficiency, construction costs can be effectively reduced and the economic benefits of the project can be improved.

[0146] As a preferred embodiment of the present application, the two methods of spatiotemporal relationship modeling and time interval calculation each have their own uses, and in most cases can be used in combination to better understand and optimize the construction process. Spatiotemporal relationship modeling can be mainly used to predict and infer possible construction conditions. For example, in the preliminary design stage of the construction plan, this method can be used to predict the possible stay time of personnel in each area, thereby helping to optimize the workflow, personnel scheduling, etc. Time interval calculation can be mainly used to track and record the actual construction process. For example, during the construction process, by recording and analyzing the actual stay time of personnel in each area, the construction plan can be adjusted in time to ensure the smooth progress of the construction process. In many cases, the two methods can be used in combination. For example, a preliminary construction plan can be formulated using spatiotemporal relationship modeling, and then the actual construction situation can be monitored using time interval calculation during the construction process, and the construction plan can be adjusted in time to achieve the best construction effect.

[0147] As a preferred embodiment of the present application, the time series analysis technology can be used to construct time series data of the stay time of personnel in different places to identify common stay patterns and abnormal behaviors. Among them, "stay pattern" can refer to the regular stay behavior or habits of personnel in various areas of the construction site, including but not limited to frequently visited areas, specific stay periods, specific lengths of stay time, etc. The identification of stay patterns helps to understand and predict the daily operations of the construction site. "Abnormal behavior" can refer to behaviors that deviate from these regular patterns, such as appearing at a time or place where they should not appear, or staying at a certain place for an abnormally long or short time. Identifying abnormal behavior can help managers promptly discover possible safety risks or construction efficiency issues.

[0148] In one embodiment, anomaly detection techniques can be used to detect anomalies in the residence time series using statistical methods such as Z-score or machine learning algorithms such as isolation forests to identify possible abnormal behaviors. These anomalies may reflect residence patterns or abnormal activities that are inconsistent with normal behaviors and are worthy of further investigation and analysis. For example, the steps of detecting anomalies in the residence time series using the Z-score method are as follows:

[0149] 1. Data preparation: Prepare the residence time series data. This can be a set of residence times arranged in chronological order, such as [10, 8, 9, 12, 15, 8, 30, 10, 8, 9], etc.

[0150] 2. Calculate the Z-score of each data point. First, calculate the mean and standard deviation of the dwell time. Then, for each dwell time, calculate its Z-score. The Z-score is a numerical value that represents the number of standard deviations from the mean. The formula is:

[0151]

[0152] Where x is the current value, μ is the mean, and σ is the standard deviation.

[0153] 3. Find outliers based on Z-score. Generally speaking, if the Z-score value of a data point is greater than a certain threshold (such as 3 or -3), we regard it as an outlier. This is because if the Z-score value of a number is far away from 0, then the number is far away from the numbers outside it.

[0154] In this example, for example, we process the sequence [10,8,9,12,15,8,30,10,8,9] and calculate the mean to be 11 and the standard deviation to be 6. According to the Z-score formula, we obtain the Z-score sequence of all residence times. For example, if one item is 30, Z-score = (30-11) / 6 = 3.17, which exceeds the threshold of 3, so this residence time is judged to be an abnormal point.

[0155] This solution comprehensively applies a variety of technical methods such as computer vision, machine learning, and data analysis, including time interval calculation, multivariate linear regression, neural networks, etc., to achieve accurate inference of personnel stay time. This comprehensive application can give full play to the advantages of various technical methods and improve the accuracy and reliability of stay time inference. By combining the activity trajectory of personnel and the spatial layout of the construction site, an accurate spatiotemporal relationship model is established. This model takes into account the comprehensive influence of multiple factors such as the stay time of personnel in different areas and work tasks, the distance between areas, and regional characteristics, so as to more accurately infer the stay time of personnel.

[0156] like Figure 4 As shown, in one embodiment, a target detection and analysis device based on multiple cameras is provided. The target detection and analysis device based on multiple cameras can be integrated into the above-mentioned computer device 120, and specifically can include a video module 510 to be detected, a motion vector acquisition module 520, a same target matching module 530, a moving trajectory acquisition module 540 and a working rule model acquisition module 550.

[0157] The video module 510 to be detected is used to obtain a number of videos to be detected collected by a number of terminal devices;

[0158] The motion vector acquisition module 520 is used to extract the image to be detected and the adjacent frame images before and after the image to be detected from the video to be detected, and obtain the motion vector of each pixel in the image to be detected compared with the adjacent frame images before and after it;

[0159] The same target matching module 530 is used to obtain all the portraits to be tested in all the videos to be tested, perform same target matching on the portraits to be tested based on facial similarity features, time features and spatial features, and cluster the portraits to be tested representing the same person to obtain a number of target persons and a set of target portraits associated with the target persons, wherein the time features are used to represent the displacement relationship of the target persons over time, and the spatial features are used to represent the spatial displacement relationship of the portrait persons;

[0160] A movement trajectory acquisition module 540 is used to obtain a movement trajectory of a target person associated with the target portrait set based on the motion vector and the target portrait set;

[0161] The work rule model acquisition module 550 is used to obtain the work rule matrix of the target person based on the target person and his / her movement trajectory and residence time in the area to be detected; and obtain the work rule model of the target person based on the work rule matrix and the work task characteristics of the target person in the area to be detected.

[0162] In the embodiments of the present application, for the explanation and description of the above-mentioned target detection and analysis device based on multiple cameras, please refer to the explanation and description of the above-mentioned corresponding method. For the description of the above-mentioned target detection and analysis method based on multiple cameras, please refer to the above text and will not be repeated here.

[0163] In the embodiment of the present application, based on the scheme executed by the present device, by recording the appearance timestamps of personnel in different areas and calculating the time intervals, a model of the personnel's residence time in each area is established. In combination with the spatial layout of the construction site, the distribution of work tasks and the regional characteristics, a spatiotemporal relationship model is established. The frequency and duration of personnel's stay in different areas are considered, and the weight of the stay time is dynamically adjusted, thereby improving the accuracy of the personnel's regional stay time inference, which brings the following advantages:

[0164] (1) Improve construction site management efficiency: By accurately inferring the length of time that various types of personnel stay in different task-type areas, construction managers can better understand the activities on the construction site, allocate resources in a timely manner, and optimize work processes, thereby improving management efficiency.

[0165] (2) Optimize the allocation of construction resources: By accurately inferring the length of stay of each person, the use of personnel and equipment can be arranged more reasonably, avoiding idleness and waste of resources and improving the utilization rate of construction resources.

[0166] (3) Improve construction safety: By real-time monitoring of personnel activities at the construction site, potential safety hazards can be discovered and resolved in a timely manner, reducing the risk of accidents and improving construction safety.

[0167] (4) Improve construction quality: By accurately inferring the residence time, the construction progress and quality can be better monitored, problems in the construction process can be discovered in time and adjustments can be made, thereby improving the construction quality.

[0168] (5) Reduce construction costs: By optimizing resource utilization and improving construction efficiency, construction costs can be effectively reduced and the economic benefits of the project can be improved.

[0169] Figure 5 The internal structure diagram of a computer device in one embodiment is shown. The computer device may specifically be Figure 1The terminal 110 (or server 120) in . As shown in 5, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected via a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system, and may also store a computer program, which, when executed by the processor, enables the processor to implement a target detection and analysis method based on multiple cameras. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to execute a target detection and analysis method based on multiple cameras. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0170] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0171] In one embodiment, the multi-camera target detection and analysis device provided in the present application can be implemented in the form of a computer program. The computer program can be Figure 5 The computer device shown in the figure is run on the computer device. The memory of the computer device can store various program modules constituting the target detection and analysis device based on multiple cameras, for example, Figure 4 The detection video module 510, motion vector acquisition module 520, same target matching module 530, movement trajectory acquisition module 540 and working rule model acquisition module 550 are shown. The computer program composed of each program module enables the processor to execute the steps of the target detection and analysis method based on multiple cameras in each embodiment of the present application described in this specification.

[0172] For example, Figure 5 The computer device shown can be Figure 4 The module 510 in the multi-camera-based target detection and analysis device shown in the figure executes step S10. The computer device can execute step S20 through the module 520. And so on.

[0173] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the target detection and analysis method based on multiple cameras as described above.

[0174] In the embodiments of the present application, please refer to the above for the description of the above-mentioned target detection and analysis method based on multiple cameras, which will not be repeated here.

[0175] In the embodiment of the present application, based on the program run by the method stored in the storage medium of the embodiment of the present application, by recording the appearance timestamps of personnel in different areas and calculating the time intervals, a model of the personnel's residence time in each area is established. In combination with the spatial layout of the construction site, the distribution of work tasks and regional characteristics, a spatiotemporal relationship model is established. The frequency and duration of personnel's stay in different areas are considered, and the weight of the residence time is dynamically adjusted, thereby improving the accuracy of the personnel's regional residence time inference, which brings the following advantages:

[0176] (1) Improve construction site management efficiency: By accurately inferring the length of time that various types of personnel stay in different task-type areas, construction managers can better understand the activities on the construction site, allocate resources in a timely manner, and optimize work processes, thereby improving management efficiency.

[0177] (2) Optimize the allocation of construction resources: By accurately inferring the length of stay of each person, the use of personnel and equipment can be arranged more reasonably, avoiding idleness and waste of resources and improving the utilization rate of construction resources.

[0178] (3) Improve construction safety: By real-time monitoring of personnel activities at the construction site, potential safety hazards can be discovered and resolved in a timely manner, reducing the risk of accidents and improving construction safety.

[0179] (4) Improve construction quality: By accurately inferring the residence time, the construction progress and quality can be better monitored, problems in the construction process can be discovered in time and adjustments can be made, thereby improving the construction quality.

[0180] (5) Reduce construction costs: By optimizing resource utilization and improving construction efficiency, construction costs can be effectively reduced and the economic benefits of the project can be improved.

[0181] In one embodiment, a multi-camera based target detection and analysis system is provided, and when the system is running, the steps of the multi-camera based target detection and analysis method as described above are executed.

[0182] In the embodiment of the present application, the system can be a computer software or program system, which runs in hardware containing a processing device, and can be deployed in a camera device or a background data processing device. When the above system is running, the corresponding method is executed. For the description of the above multi-camera target detection and analysis method, please refer to the above text and will not be repeated here.

[0183] In the embodiment of the present application, the system establishes a model of the personnel's residence time in each area by recording the timestamps of the personnel's appearance in different areas and calculating the time intervals. The system also establishes a spatiotemporal relationship model by combining the spatial layout of the construction site, the distribution of work tasks, and the regional characteristics. The system considers the frequency and duration of personnel's stay in different areas, dynamically adjusts the weight of the stay time, and improves the accuracy of personnel's regional stay time inference, which brings the following advantages:

[0184] (1) Improve construction site management efficiency: By accurately inferring the length of time that various types of personnel stay in different task-type areas, construction managers can better understand the activities on the construction site, allocate resources in a timely manner, and optimize work processes, thereby improving management efficiency.

[0185] (2) Optimize the allocation of construction resources: By accurately inferring the length of stay of each person, the use of personnel and equipment can be arranged more reasonably, avoiding idleness and waste of resources and improving the utilization rate of construction resources.

[0186] (3) Improve construction safety: By real-time monitoring of personnel activities at the construction site, potential safety hazards can be discovered and resolved in a timely manner, reducing the risk of accidents and improving construction safety.

[0187] (4) Improve construction quality: By accurately inferring the residence time, the construction progress and quality can be better monitored, problems in the construction process can be discovered in time and adjustments can be made, thereby improving the construction quality.

[0188] (5) Reduce construction costs: By optimizing resource utilization and improving construction efficiency, construction costs can be effectively reduced and the economic benefits of the project can be improved.

[0189] In one embodiment of the present application, a robot is provided, on which a plurality of monitoring cameras are provided. The monitoring cameras perform regional monitoring by executing the target detection and analysis method based on multiple cameras as described above.

[0190] In an embodiment of the present application, there may be one or more monitoring robots installed in a site, and the number of cameras installed on each robot may be one or more. All cameras on multiple monitoring robots can be networked in a unified manner, and the data can be input into the same-day analysis system, and target detection analysis can be performed based on the above scheme. Based on the monitoring robot of the embodiment of the present application, the appearance timestamps of personnel in different areas can be recorded and the time intervals can be calculated. A residence time model of personnel in various areas is established. In combination with the spatial layout of the construction site, the distribution of work tasks and regional characteristics, a spatiotemporal relationship model is established. The frequency and duration of personnel's stay in different areas are considered, and the weight of the residence time is dynamically adjusted, thereby improving the accuracy of the inference of the personnel's regional residence time, bringing the following advantages:

[0191] (1) Improve construction site management efficiency: By accurately inferring the length of time that various types of personnel stay in different task-type areas, construction managers can better understand the activities on the construction site, allocate resources in a timely manner, and optimize work processes, thereby improving management efficiency.

[0192] (2) Optimize the allocation of construction resources: By accurately inferring the length of stay of each person, the use of personnel and equipment can be arranged more reasonably, avoiding idleness and waste of resources and improving the utilization rate of construction resources.

[0193] (3) Improve construction safety: By real-time monitoring of personnel activities at the construction site, potential safety hazards can be discovered and resolved in a timely manner, reducing the risk of accidents and improving construction safety.

[0194] (4) Improve construction quality: By accurately inferring the residence time, the construction progress and quality can be better monitored, problems in the construction process can be discovered in time and adjustments can be made, thereby improving the construction quality.

[0195] (5) Reduce construction costs: By optimizing resource utilization and improving construction efficiency, construction costs can be effectively reduced and the economic benefits of the project can be improved.

[0196] It should be understood that, although each step in the flow chart of each embodiment of the present application is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0197] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0198] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0199] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A target detection and analysis method based on multiple cameras, characterized in that: The method comprises: Acquire a number of videos to be detected collected by a number of terminal devices; Extracting an image to be detected and adjacent frame images before and after the image to be detected from the video to be detected, and obtaining a motion vector of each pixel in the image to be detected compared with the adjacent frame images before and after the image to be detected; Acquire all the portraits to be tested in all the videos to be tested, perform the same target matching on the portraits to be tested based on the facial similarity feature, the time feature and the spatial feature, and cluster the portraits to be tested representing the same person to obtain several target persons and a set of target portraits associated with the target persons, wherein the time feature is used to represent the displacement relationship of the target person over time, and the spatial feature is used to represent the spatial displacement relationship of the portrait persons; Based on the motion vector and the target portrait set, a moving trajectory of the target person associated with the target portrait set is obtained; Based on the target person and his / her moving trajectory and residence time in the area to be detected, a work rule matrix of the target person is obtained; based on the work rule matrix and the work task characteristics of the target person in the area to be detected, a work rule model of the target person is obtained; The method for obtaining the work rule matrix of the target personnel is as follows: Divide the area to be detected into several sub-areas, and perform three-dimensional modeling on the sub-areas to obtain the spatial position coordinates of each sub-area; Obtain the target person's residence time in each sub-area and the movement sequence of entering each sub-area; The work rule matrix includes the target personnel's appearance frequency matrix F in each sub-area, the transition probability matrix P between work tasks, and the work time matrix W in each sub-area; Based on the residence time and the movement sequence, the occurrence frequency matrix F, the transition probability matrix P and the working duration matrix W are obtained; The working rule model is obtained as follows: Obtain the work task characteristics W of the target personnel in each sub-area i , as well as the distance matrix D and characteristic matrix S that characterize the relative positions between each sub-area, to obtain the spatiotemporal relationship function of personnel movement; Based on the time-space relationship function of the target personnel, the work rule model of the target personnel is obtained; The method further comprises: Acquiring dynamic weight parameters, wherein the dynamic weight parameters at least include a stay frequency calculation or a duration calculation; Based on the dynamic weight parameter, the weight of the working rule matrix is ​​dynamically adjusted.

2. The target detection and analysis method based on multiple cameras according to claim 1 is characterized in that: The method for matching the same target based on face similarity features is: Acquire image data from the video to be detected, and perform face detection on the image data to obtain a face image; Extracting features from the face image to obtain a feature vector; Combining the feature vectors corresponding to all the facial images obtained in the video to be detected to obtain a vector library; Perform similarity matching calculation on the feature vector corresponding to the face image of the target person and the feature vector in the vector library to obtain a similarity value; The portraits to be tested corresponding to a number of feature vectors whose similarity values ​​are higher than a similarity threshold are set as a target portrait set of the target person; The method of extracting features from the face image to obtain a feature vector is: Performing face cropping and key feature point detection on the face image; Aligning the key feature points based on the detection result of the key feature point detection, and masking the area outside the face in the face image to obtain a preprocessed image; A deep learning model is used to extract features from the preprocessed image to obtain a feature vector.

3. The target detection and analysis method based on multiple cameras according to claim 1 is characterized in that: The method for obtaining the pixel motion vector is: Assuming that at time t, the position of a pixel is (x, y), and at time t+1, its position is (x', y'), then the motion vector (u, v) is calculated as: Among them, (u,v) represents the t The motion vector of the pixel point (x, y) in , min means taking the minimum value.

4. A target detection and analysis device based on multiple cameras, characterized in that: The target detection and analysis device based on multiple cameras comprises: The video module to be detected is used to obtain a number of videos to be detected collected by a number of terminal devices; A motion vector acquisition module is used to extract the image to be detected and the adjacent frame images before and after the image to be detected from the video to be detected, and obtain the motion vector of each pixel in the image to be detected compared with the adjacent frame images before and after it; The same target matching module is used to obtain all the portraits to be tested in all the videos to be tested, perform same target matching on the portraits to be tested based on facial similarity features, time features and spatial features, and cluster the portraits to be tested that represent the same person to obtain several target persons and a set of target portraits associated with the target persons. The time features are used to represent the displacement relationship of the target persons over time, and the spatial features are used to represent the spatial displacement relationship of the portrait persons. A movement trajectory acquisition module, used for obtaining the movement trajectory of the target person associated with the target portrait set based on the motion vector and the target portrait set; A work rule model acquisition module is used to obtain a work rule matrix of the target person based on the target person and his / her movement trajectory and residence time in the area to be detected; and obtain a work rule model of the target person based on the work rule matrix and the work task characteristics of the target person in the area to be detected; The method for obtaining the work rule matrix of the target personnel is as follows: Divide the area to be detected into several sub-areas, and perform three-dimensional modeling on the sub-areas to obtain the spatial position coordinates of each sub-area; Obtain the target person's residence time in each sub-area and the movement sequence of entering each sub-area; The work rule matrix includes the target personnel's appearance frequency matrix F in each sub-area, the transition probability matrix P between work tasks, and the work time matrix W in each sub-area; Based on the residence time and the movement sequence, the occurrence frequency matrix F, the transition probability matrix P and the working duration matrix W are obtained; The working rule model is obtained as follows: Obtain the work task characteristics W of the target personnel in each sub-area i , as well as the distance matrix D and characteristic matrix S that characterize the relative positions between each sub-area, to obtain the spatiotemporal relationship function of personnel movement; Based on the time-space relationship function of the target personnel, the work rule model of the target personnel is obtained; The method further comprises: Acquiring dynamic weight parameters, wherein the dynamic weight parameters at least include a stay frequency calculation or a duration calculation; Based on the dynamic weight parameter, the weight of the working rule matrix is ​​dynamically adjusted.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the target detection and analysis method based on multiple cameras as described in any one of claims 1 to 3.

6. A robot, characterized in that: The robot is provided with a plurality of monitoring cameras, and the monitoring cameras perform regional monitoring by executing the target detection and analysis method based on multiple cameras as described in any one of claims 1 to 3.

Images