A tracking and positioning system based on image data and a method of use

Abstract

A tracking and positioning system based on image data and a method of use, comprising: multiple cameras, an identity recognition server and a tracking and positioning system server. The image processing algorithm is used to obtain the direction data of the five dimensions of eyes, head, shoulders, hands and feet, which enhances the accuracy and rationality of the judgment of the current direction of the personnel and also improves the efficiency of direction judgment; and, by adding technical means such as the floor tile sensing direction data and action route of the personnel, the big data means are used to assist in obtaining the predicted movement direction of the key observation personnel, so that the prediction of the rotation and movement direction of the key observation personnel is more accurate.

Description

Technical Field

[0001] The present invention belongs to the field of tracking and positioning, and in particular relates to a tracking and positioning system based on image data and a use method thereof. Background Art

[0002] The existing tracking and positioning technology of image data considers relatively single analysis dimensions and image factors, resulting in certain deviations in the capture of human actions and the prediction of the next movement direction. Summary of the invention

[0003] The purpose of the present invention is to provide a tracking and positioning system based on image data and a method of use to solve the technical problems existing in the prior art.

[0004] A tracking and positioning system based on image data, comprising:

[0005] Multiple cameras: obtain facial and full-body images of all people in the monitoring area;

[0006] Identity recognition server: after processing the received facial image data of the person, perform identity recognition based on the facial image data; mark the key observation persons, and send the full body image data of the key observation persons to the tracking and positioning system server;

[0007] Tracking and positioning system server: Based on the acquired full-body image data of the key observed person, the image processing algorithm is used to decompose the person's image data into eye image (A), head image (B), shoulder image (C), hand image (D), and foot image (E); the tracking and positioning system server processes the last three frames of the eye image, head image, shoulder image, hand image, and foot image respectively; obtains the key observed person's eye sight data, head orientation data, shoulder orientation data, hand waving direction data, and foot orientation data; based on the eye sight data (Y), head orientation data (T), shoulder orientation data (J), hand waving direction data (S), and foot orientation data (X), the current direction (F) of the key observed person is determined;

[0008] Perform weighted superposition on the eye sight data (Y), head orientation data (T), shoulder orientation data (J), hand waving direction data (S), and foot orientation data (X) to obtain the current direction (F) of the key observed person;

[0009] The tracking and positioning system server obtains the head rotation data, shoulder rotation data, foot rotation and movement data through a machine learning algorithm based on the received complete head image, shoulder image and foot image of the key observation person; the tracking and positioning system server calculates the predicted rotation and movement direction of the key observation person based on the head rotation data, shoulder rotation data, foot rotation and movement data; the tracking and positioning system server determines the predicted movement direction of the key observation person based on the current direction and the predicted rotation and movement direction of the key observation person;

[0010] The tracking and positioning system server combines the floor tile sensing direction to correct the predicted rotation and movement direction of the key observation personnel;

[0011] The tracking and positioning system server determines whether the key observation person is currently in a stationary or moving state: when the key observation person is in a stationary state, the predicted rotation and movement direction of the key observation person is obtained based on the head rotation data; when the key observation person is in a moving state, the predicted rotation and movement direction of the key observation person is obtained based on the shoulder rotation data, foot rotation and movement data.

[0012] Preferably, the method for using the tracking and positioning system based on image data includes:

[0013] Step S100: A person enters the monitoring area;

[0014] Step S200: multiple cameras acquire facial images of all persons in the monitoring area;

[0015] Step S300: After the identity recognition server processes the received facial image data of the person, it performs identity recognition of the person based on the facial image data;

[0016] Step S350: the identity recognition server marks the key observation person and sends the full body image data of the key observation person to the tracking and positioning system server;

[0017] Step S400: The tracking and positioning system server uses an image processing algorithm, such as a ROI algorithm, based on the acquired full-body image data of the key observed person to decompose the person image data into eye images (A), head images (B), shoulder images (C), hand images (D), and foot images (E);

[0018] Step S500: The tracking and positioning system server processes the last three frames of the eye image, the head image, the shoulder image, the hand image, and the foot image respectively; and obtains the eye sight data, the head orientation data, the shoulder orientation data, the hand waving direction data, and the foot orientation data of the key observed person;

[0019] Step S600: Based on the eye sight data (Y), the head orientation data (T), the shoulder orientation data (J), the hand waving direction data (S), and the foot orientation data (X), determine the current direction (F) of the key observation person.

[0020] Preferably, the method for using the tracking and positioning system based on image data also includes: weighted superposition of eye sight data (Y), head orientation data (T), shoulder orientation data (J), hand waving direction data (S), and foot orientation data (X) to obtain the current direction (F) of the key observed person.

[0021] Preferably, the method for using the tracking and positioning system based on image data also includes:

[0022] Step S700: The tracking and positioning system server obtains head rotation data, shoulder rotation data, foot rotation and movement data through a machine learning algorithm based on the received complete head image, shoulder image, and foot image of the key observed person;

[0023] Step S800: The tracking and positioning system server calculates the predicted rotation and movement direction of the key observed person based on the head rotation data, shoulder rotation data, foot rotation and movement data;

[0024] Step S900: The tracking and positioning system server determines the predicted movement direction of the key observed person based on the current direction and the predicted rotation and movement direction of the key observed person.

[0025] Preferably, the method for using the tracking and positioning system based on image data also includes:

[0026] Step S810: The tracking and positioning system server combines the floor tile sensing direction to correct the predicted rotation and movement direction of the key observation person.

[0027] Preferably, the method for using the tracking and positioning system based on image data also includes:

[0028] Step S800: The tracking and positioning system server obtains head rotation data, shoulder rotation data, foot rotation and movement data;

[0029] Step S810: The tracking and positioning system server determines whether the key observed person is currently in a stationary state or a moving state: when the key observed person is in a stationary state, jump to step S811; when the key observed person is in a moving state, jump to step S812;

[0030] Step S811: Obtain the predicted rotation and movement direction of the key observed person based on the head rotation data.

[0031] Step S812: Obtain the predicted rotation and movement direction of the key observation person based on the shoulder rotation data, the foot rotation and movement data.

[0032] The tracking and positioning system based on image data and the method of use claimed in the present invention can be applied to large outdoor places such as stations, squares, airports, hospitals, and schools. The image processing algorithm is used to obtain direction data in five dimensions of eyes, head, shoulders, hands, and feet, thereby enhancing the accuracy and rationality of the judgment of the current direction of the person; at the same time, the weighted superposition priority of the five dimensions of eyes, head, shoulders, hands, and feet is distinguished, thereby making the judgment of the current orientation of the person more accurate and reasonable, and also improving the efficiency of direction judgment; and, by adding technical means such as the floor tile sensing direction data and action route of the person, big data means are used to assist in deriving the predicted movement direction of the key observation person, so that the prediction of the rotation and movement direction of the key observation person is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0034] Figure 1 Schematic diagram of the structure of the tracking and positioning system based on image data.

[0035] Figure 2 This is a structural block diagram of the camera of the tracking and positioning system based on image data.

[0036] Figure 3 This is a structural block diagram of the tracking and positioning system server based on image data.

[0037] Figure 4 The present invention is a flowchart of a method for using a tracking and positioning system based on image data.

[0038] Figure 5 Schematic diagram of the decomposition areas of the whole body image.

[0039] Figure 6 This is a weighted overlay diagram of the current direction of key observation personnel.

[0040] Figure 7 Flowchart of steps for personnel identification.

[0041] Figure 8 Flowchart of predicted movement direction for key observation personnel.

[0042] Fig. 9 It is a schematic diagram of the three composite dimensions of head rotation, shoulder rotation, foot rotation and movement.

[0043] Fig.10 Schematic diagram of the weight sensing floor tile system.

[0044] Fig.11 This is a schematic diagram of the movement routes of key observation personnel.

[0045] Fig.12 The present invention is a flow chart of another embodiment of predicting the movement direction of a key observed person. DETAILED DESCRIPTION

[0046] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of use of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0047] As the instruction manual Figure 1 As shown: a tracking and positioning system based on image data, including: multiple cameras, an identity recognition server, and a tracking and positioning system server; the cameras are used to collect facial images and full-body images of people, and the cameras include fixed cameras, movable cameras, drone cameras, mobile robot cameras, etc.; the identity recognition server identifies the identity information of people based on face recognition technology, and transmits full-body image data to the tracking and positioning system server; the tracking and positioning system server is used to focus on observing people's motion capture, posture judgment and movement direction prediction.

[0048] As the instruction manual Figure 2 As shown: the camera includes: optical elements, a control unit, a memory, and a data transmission unit; the optical elements are used to generate facial images and full-body images of people; the control unit is used for process management of optical images; the memory is used to store image data and full-body images; and the data transmission unit is used for network information transmission.

[0049] As the instruction manual Figure 3 As shown: the identity recognition server includes: a control unit, a large-capacity storage unit, a display unit and a data transmission unit; the control unit is used for identity recognition process control and full-body image data transmission process control; the large-capacity storage unit is used to store personnel face images, key observation personnel information and key monitoring personnel information; key monitoring personnel include suspicious persons, criminal suspects, wanted criminals, blacklisted persons, etc.; when the face image of a person matches the face image of a key monitoring person, the identity recognition server determines that the person is a key observation person; the display unit is used to display the identity recognition result and full-body image transmission process information; the data transmission unit is used for network information transmission.

[0050] The tracking and positioning system server includes: a control unit, a large-capacity storage unit, a display unit and a data transmission unit; the control unit is used for motion capture, posture judgment, movement direction prediction and data transmission process control; the large-capacity storage unit is used to store control process information, full-body image information and comparison information, etc.; the display unit is used to display information such as the action route and predicted action direction of the key observation personnel; the data transmission unit is used for network information transmission.

[0051] As the instruction manual Figure 4 As shown in one embodiment, a method for using a tracking and positioning system based on image data includes:

[0052] Step S100: A person enters the monitoring area;

[0053] Step S200: multiple cameras acquire facial images of all persons in the monitoring area;

[0054] Step S300: After the identity recognition server processes the received facial image data of the person, it performs identity recognition of the person based on the facial image data;

[0055] Step S350: the identity recognition server marks the key observation person and sends the full body image data of the key observation person to the tracking and positioning system server;

[0056] Step S400: The tracking and positioning system server uses an image processing algorithm, such as a ROI algorithm, based on the acquired full-body image data of the key observed person to decompose the person image data into eye image A, head image B, shoulder image C, hand image D, and foot image E;

[0057] Step S500: The tracking and positioning system server processes the last three frames of the eye image, the head image, the shoulder image, the hand image, and the foot image respectively; and obtains the eye sight data, the head orientation data, the shoulder orientation data, the hand waving direction data, and the foot orientation data of the key observed person;

[0058] Step S600: Based on the eye sight data Y, the head orientation data T, the shoulder orientation data J, the hand waving direction data S, and the foot orientation data X, determine the current direction F of the key observed person.

[0059] In this embodiment, an image processing algorithm is used to obtain direction data in five dimensions: eyes, head, shoulders, hands, and feet, thereby enhancing the accuracy and rationality of the judgment of the current direction of the person.

[0060] As the instruction manual Figure 5 As shown: the whole body image data of the key observation person is decomposed into eye image data A, head image data B, shoulder image data C, hand image data D, and foot image data E.

[0061] In one embodiment, a method for using a tracking and positioning system based on image data includes: step S300: after the identity recognition server processes the received facial image data of the person, it uses HOG (Histogram of Oriented Gridients, feature detection algorithm), LBP (Local Binary Patterns, local binary pattern algorithm), and Gabor filtering algorithm to extract the head / eye / nose / mouth features of the person; based on the convolutional neural network algorithm (CNN, Convolutional Neural Network) and facial landmark detection, the person's identity is recognized, thereby improving the diversity and accuracy of identity recognition.

[0062] As the instruction manual Figure 6 As shown: the eye sight vector Y, the head orientation vector T, the shoulder orientation vector J, the hand waving direction vector S, and the foot orientation vector X are weighted and superimposed to obtain the current direction F of the key observed person; the first gear of the direction vector superposition weight: the foot orientation vector X and the shoulder orientation vector J, the second gear of the direction vector superposition weight: the head orientation vector T, the third gear of the direction vector superposition weight: the eye sight vector Y, the fourth gear of the direction vector superposition weight: the hand waving direction vector S, the weight size: the first gear> the second gear> the third gear> the fourth gear. In this embodiment, the weighted superposition priority of the five dimensions of eyes, head, shoulders, hands, and feet is distinguished, so that the determination of the current direction of the person is more accurate and reasonable, and the efficiency of direction judgment is also improved.

[0063] As the instruction manual Figure 7 As shown in one embodiment, a method for using a tracking and positioning system based on image data includes:

[0064] Step S300: After the identity recognition server processes the received facial image data of the person, it uses ROI (region of interest) technology to identify the facial contour / eyes / nose / ears / mouth of the person;

[0065] Step S310: Using feature extraction and feature classification technology, obtain feature point information of the person's facial contour / eyes / nose / ears / mouth, and calculate the similarity and distance of the feature points;

[0066] Step S320: convert the facial image data and the calculation results of the feature points into byte stream (IO stream) data, and compare them with the facial image byte stream data pre-stored in the key monitoring personnel database. The pre-stored facial images of the key monitoring personnel and the identity file information of the corresponding personnel are all stored in the key monitoring personnel database. The comparison algorithm includes one or more of the following algorithms: convolutional neural network algorithm, Eigenfaces algorithm, Haar Cascade algorithm.

[0067] In this embodiment, the efficiency and accuracy of the personnel identification step are enhanced through the design and combination of artificial intelligence algorithms.

[0068] As the instruction manual Figure 8 As shown in one embodiment, a method for using a tracking and positioning system based on image data includes:

[0069] Step S700: The tracking and positioning system server obtains head rotation data, shoulder rotation data, foot rotation and movement data through a machine learning algorithm based on the received complete head image, shoulder image, and foot image of the key observed person;

[0070] Step S800: The tracking and positioning system server calculates the predicted rotation and movement direction of the key observed person based on the head rotation data, shoulder rotation data, foot rotation and movement data;

[0071] Step S900: The tracking and positioning system server determines the predicted movement direction of the key observed person based on the current direction and the predicted rotation and movement direction of the key observed person.

[0072] In this embodiment, on the basis of obtaining the current direction of the key observation person, the predicted movement direction of the key observation person is obtained by combining the superposition of three composite dimensions, namely, head rotation, shoulder rotation, foot rotation and movement, thereby expanding the function of the tracking and positioning system.

[0073] As the instruction manual Fig. 9 As shown in the figure: the complete head image of the key observation person received is the video data at one end, such as the video data with a time length of 3S. The tracking and positioning system server obtains the head rotation data based on the video data. T The complete shoulder image of the key observation person received is the video data at one end, such as the video data with a time length of 3S. The tracking and positioning system server obtains the shoulder rotation data based on the video data. J The complete foot image of the key observation person received is the video data at one end, such as the video data with a time length of 3S. The tracking and positioning system server obtains the foot rotation and movement data based on the video data. X .

[0074] In this embodiment, data of three composite dimensions, namely, head rotation, shoulder rotation, foot rotation and movement, are extracted from video data of a certain length, and big data technology is combined to make the data of the three composite dimensions accurate.

[0075] In one embodiment, a method for using a tracking and positioning system based on image data includes:

[0076] Step S100: A person enters a monitoring area, where weight-sensing floor tiles are laid;

[0077] Step S810: The tracking and positioning system server combines the floor tile sensing direction to correct the predicted rotation and movement direction of the key observation person.

[0078] In this embodiment, the technical means and data dimensions are broadened, and weight sensing technical means are combined to further correct the predicted rotation and movement direction of the key observation person.

[0079] As the instruction manual Fig.10 As shown: Each weight-sensing floor tile is equipped with multiple weight sensors. If a person steps on a floor tile with one foot, Fig.10 As shown in A, the ten weight sensors of the floor tiles can sense the tilt direction and degree of the foot; if a person steps on two floor tiles with one foot, Fig.10 As shown in B, the tracking and positioning system server can sense the tilt direction and degree of the foot based on the six weight sensors A2~A4 and B6~B8; if a person steps on four tiles with one foot, Fig.10 As shown in C, the tracking and positioning system server can sense the tilt direction and degree of the foot based on twelve weight sensors A3~A5, B5~B7, C1~C3 and D1, D7~D8.

[0080] The tracking and positioning system server identifies the person's two feet based on the person's full-body image, and combines the inclination direction and degree of the two feet. Based on a large amount of data on the inclination direction and degree of the two feet and the person's floor tile sensing direction data at the time, the server obtains the person's current floor tile sensing direction through a deep learning algorithm.

[0081] Reference Manual Attached Fig.11 As shown in one embodiment, a method for using a tracking and positioning system based on image data includes:

[0082] Step S350: the identity recognition server marks the key observation person and sends the full body image data of the key observation person to the tracking and positioning system server;

[0083] Step S351: The tracking and positioning system server starts recording the floor tile sensing direction data of the key observation person and fitting it into the action route of the key observation person. During this process, the current direction, rotation and movement direction of the key observation person are combined to form a data set.

[0084] Step S900: The tracking and positioning system server determines the predicted movement direction of the key observation person based on the key observation person's action route, current direction, and predicted rotation and movement direction.

[0085] In this embodiment, the action route of the key observation person is combined with the current direction, rotation and movement direction of the key observation person to form a data set, and big data means are used to assist in deriving the predicted movement direction of the key observation person.

[0086] As the instruction manual Fig.12 As shown in one embodiment, a method for using a tracking and positioning system based on image data includes:

[0087] Step S800: The tracking and positioning system server obtains head rotation data, shoulder rotation data, foot rotation and movement data;

[0088] Step S810: The tracking and positioning system server determines whether the key observed person is currently in a stationary state or a moving state: when the key observed person is in a stationary state, jump to step S811; when the key observed person is in a moving state, jump to step S812;

[0089] Step S811: Obtain the predicted rotation and movement direction of the key observed person based on the head rotation data.

[0090] Step S812: Obtain the predicted rotation and movement direction of the key observation person based on the shoulder rotation data, the foot rotation and movement data.

[0091] In this embodiment, when the movement state of a person is different, the orientation of different body parts has different degrees of influence on the prediction of the rotation and movement direction. Taking this into consideration, by distinguishing between the static or moving states of the person, the prediction of the rotation and movement direction of the person being observed can be made more accurate.

[0092] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A tracking and positioning system based on image data, characterized in that: include: Multiple cameras: obtain facial and full-body images of all people in the monitoring area; Identity recognition server: after processing the received facial image data of the person, perform identity recognition based on the facial image data; mark the key observation persons, and send the full body image data of the key observation persons to the tracking and positioning system server; Tracking and positioning system server: Based on the acquired full-body image data of the key observed person, the image processing algorithm is used to decompose the person's image data into eye image (A), head image (B), shoulder image (C), hand image (D), and foot image (E); the tracking and positioning system server processes the last three frames of the eye image, head image, shoulder image, hand image, and foot image respectively; obtains the key observed person's eye sight data, head orientation data, shoulder orientation data, hand waving direction data, and foot orientation data; based on the eye sight data (Y), head orientation data (T), shoulder orientation data (J), hand waving direction data (S), and foot orientation data (X), the current direction (F) of the key observed person is determined; Perform weighted superposition on the eye sight data (Y), head orientation data (T), shoulder orientation data (J), hand waving direction data (S), and foot orientation data (X) to obtain the current direction (F) of the key observed person; The tracking and positioning system server obtains the head rotation data, shoulder rotation data, foot rotation and movement data through the machine learning algorithm based on the received complete head image, shoulder image and foot image of the key observation person; the tracking and positioning system server calculates the predicted rotation and movement direction of the key observation person based on the head rotation data, shoulder rotation data, foot rotation and movement data; the tracking and positioning system server determines the predicted movement direction of the key observation person based on the current direction of the key observation person and the predicted rotation and movement direction; the tracking and positioning system server corrects the predicted rotation and movement direction of the key observation person in combination with the floor tile sensing direction; The tracking and positioning system server determines whether the key observation person is currently in a stationary or moving state: when the key observation person is in a stationary state, the predicted rotation and movement direction of the key observation person is obtained based on the head rotation data; when the key observation person is in a moving state, the predicted rotation and movement direction of the key observation person is obtained based on the shoulder rotation data, foot rotation and movement data.

2. The method for using the tracking and positioning system based on image data as claimed in claim 1, characterized in that: include: Step S100: A person enters the monitoring area; Step S200: multiple cameras acquire facial images of all persons in the monitoring area; Step S300: After the identity recognition server processes the received facial image data of the person, it performs identity recognition of the person based on the facial image data; Step S350: the identity recognition server marks the key observation person and sends the full body image data of the key observation person to the tracking and positioning system server; Step S400: The tracking and positioning system server uses an image processing algorithm, such as a ROI algorithm, based on the acquired full-body image data of the key observed person to decompose the person image data into eye images (A), head images (B), shoulder images (C), hand images (D), and foot images (E); Step S500: The tracking and positioning system server processes the last three frames of the eye image, the head image, the shoulder image, the hand image, and the foot image respectively; and obtains the eye sight data, the head orientation data, the shoulder orientation data, the hand waving direction data, and the foot orientation data of the key observed person; Step S600: Based on the eye sight data (Y), the head orientation data (T), the shoulder orientation data (J), the hand waving direction data (S), and the foot orientation data (X), determine the current direction (F) of the key observation person.

3. The method for using the tracking and positioning system according to claim 2, wherein: include: The eye sight data (Y), head orientation data (T), shoulder orientation data (J), hand waving direction data (S), and foot orientation data (X) are weightedly superimposed to obtain the current direction (F) of the key observed person.

4. The method for using the tracking and positioning system according to claim 3, wherein: include: Step S700: The tracking and positioning system server obtains head rotation data, shoulder rotation data, foot rotation and movement data through a machine learning algorithm based on the received complete head image, shoulder image, and foot image of the key observed person; Step S800: The tracking and positioning system server calculates the predicted rotation and movement direction of the key observed person based on the head rotation data, shoulder rotation data, foot rotation and movement data; Step S900: The tracking and positioning system server determines the predicted movement direction of the key observed person based on the current direction and the predicted rotation and movement direction of the key observed person.

5. The method for using the tracking and positioning system according to claim 4, wherein: include: Step S810: The tracking and positioning system server combines the floor tile sensing direction to correct the predicted rotation and movement direction of the key observation person.

6. The method for using the tracking and positioning system according to claim 4, wherein: include: Step S800: The tracking and positioning system server obtains head rotation data, shoulder rotation data, foot rotation and movement data; Step S810: The tracking and positioning system server determines whether the key observed person is currently in a stationary state or a moving state: when the key observed person is in a stationary state, jump to step S811; when the key observed person is in a moving state, jump to step S812; Step S811: obtaining the predicted rotation and movement direction of the key observed person based on the head rotation data; Step S812: Obtain the predicted rotation and movement direction of the key observation person based on the shoulder rotation data, the foot rotation and movement data.

Images