Personnel positioning method and system based on image analysis
Through the personnel positioning method based on image analysis, the training set and prediction model are used to combine camera internal reference and position information to calculate the actual angle and coordinate of the personnel, and solve the high-cost personnel positioning problem in the existing technology, achieving low cost and high accuracy positioning effect.
Patent Information
- Application Number
- CN202411292083.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-14
AI Technical Summary
In the prior art, achieving personnel positioning through binocular cameras and depth of field cameras requires additional hardware support, resulting in high costs and lack of low-cost solutions.
Using the personnel positioning method based on image analysis, the training set is constructed by obtaining the actual height of the person and the original image, the object detection model is trained, and the prediction box is marked with the prediction model, and the actual angle and coordinate of the target personnel are calculated based on the internal reference and position information of the camera. , realize personnel positioning.
No need to use a binocular camera or depth of field camera, just a normal camera can achieve personnel positioning, reducing costs and improving positioning accuracy and usability.
Smart Images

Figure CN118799404B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of personnel positioning, and in particular to a personnel positioning method and system based on image analysis. Background Art
[0002] In some factories or construction sites where the working environment is complex, there are many sources of danger, and the mobility of personnel is high, the positioning technology of the staff is particularly important. At present, both binocular cameras and depth-of-field cameras can realize the function of camera positioning. Binocular camera technology simulates the function of the human eye, uses two cameras to capture images from different angles, and calculates the three-dimensional spatial position of the personnel through image processing algorithms. Depth-of-field camera technology obtains the depth information of the personnel by measuring the time or phase difference of light propagation inside the camera to realize the positioning of the personnel.
[0003] However, using binocular cameras and depth-of-field cameras to achieve personnel positioning usually requires additional hardware support and must use dedicated cameras, which greatly increases the cost. Therefore, how to reduce the cost of using cameras to achieve personnel positioning is an urgent problem to be solved. Summary of the invention
[0004] In order to reduce the cost of using cameras to locate personnel, the present application provides a personnel positioning method and system based on image analysis.
[0005] In the first aspect, the present application provides a personnel positioning method based on image analysis, which adopts the following technical solution:
[0006] A personnel positioning method based on image analysis, comprising:
[0007] Obtaining the actual height of the person and the original image, and constructing a training set based on the actual height of the person and the image of the person; the original image is an image taken by a camera and containing the person;
[0008] Use the training set to train the preset target detection model to obtain a prediction model;
[0009] Acquire an image of a person to be located, input the image of the person to be located into the prediction model, and mark a prediction frame in the image of the person to be located by using the prediction model; wherein the image of the person to be located contains preset known grid points; the prediction frame is used to mark the target person in the image of the person to be located;
[0010] Obtaining the internal parameters and location information of the camera; wherein the internal parameters include the size, resolution and focal length of the camera sensor;
[0011] Obtaining an actual horizontal angle and an actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal reference and the location information;
[0012] According to the actual horizontal angle, the actual vertical angle, the internal parameters, the location information, the predicted frame of the image of the person to be located, and the known grid points, the relative position of the target person and the camera is analyzed to generate the actual coordinates of the target person.
[0013] Optionally, the obtaining of the actual height of the person and the original image, and constructing a training set according to the actual height of the person and the image of the person, specifically includes:
[0014] Calculating the ratio between the actual height of the person and the preset standard height;
[0015] Scaling the initial target frame according to the ratio value to obtain a scaled target frame of the person;
[0016] Annotating the zoom target frame in the original image and calibrating the type label of each zoom target frame to obtain a sample image;
[0017] A training set is constructed based on a number of the sample images.
[0018] Optionally, the method of using the training set to train a preset target detection model to obtain a prediction model specifically includes:
[0019] Input each sample image in the training set into the target detection model, and output the prior frame through the target detection model;
[0020] Match the prior frame and the scaled target frame to obtain the matching result, and calculate the positioning loss function of the target detection model based on the matching result;
[0021] The model parameters of the target detection model are iteratively updated according to the positioning loss function to obtain the prediction model.
[0022] Optionally, obtaining the actual horizontal angle and the actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal reference and the location information specifically includes:
[0023] Generate a horizontal field of view angle and a vertical field of view angle of the camera according to the size of the sensor and the focal length;
[0024] Generate an imaging center point of the prediction box according to the imaging coordinates of the prediction box;
[0025] Obtaining a first horizontal offset angle and a first vertical offset angle between the prediction frame and the camera center point according to the imaging center point, the horizontal field of view angle, the vertical field of view angle, and the resolution;
[0026] Get the second horizontal offset angle and the second vertical offset angle between the camera and the world coordinate system;
[0027] Obtaining an actual horizontal angle of the target person according to the first horizontal offset angle and the second horizontal offset angle;
[0028] The actual vertical angle of the target person is obtained according to the first vertical offset angle and the second vertical offset angle.
[0029] Optionally, the analyzing the relative position of the target person and the camera according to the actual horizontal angle, the actual vertical angle, the internal parameter, the location information, the predicted frame of the image of the person to be located, and the known grid points to generate the actual coordinates of the target person specifically includes:
[0030] According to the coordinates of the known grid points in the image of the person to be located, the prediction box is projected and transformed to generate the projection coordinates of the prediction box, and the actual length of the imaging part of the target person on the camera is obtained by analyzing the projection coordinates;
[0031] Generate an actual distance value between the target person and the camera according to the actual length and the internal parameters of the camera;
[0032] Calculate the relative coordinates of the target person relative to the camera based on the actual distance value, the actual horizontal angle and the actual vertical angle;
[0033] The relative coordinates are superimposed with the location information of the camera to obtain the actual coordinates of the target person.
[0034] Optionally, the projecting transformation of the prediction box to generate the projection coordinates of the prediction box according to the coordinates of the known grid points in the image of the person to be located, and obtaining the actual length of the imaging portion of the target person on the camera by analyzing the projection coordinates, specifically includes:
[0035] Acquire several known grid points around the target person in the image to be located as target grid points;
[0036] Obtain the perspective change coordinates and imaging coordinates of each target grid point;
[0037] Generate a perspective change matrix of the camera according to the perspective change coordinates and imaging coordinates of a plurality of the target grid points;
[0038] According to the perspective change matrix, the imaging coordinates of the prediction box are projected and transformed to obtain the projection coordinates of the prediction box;
[0039] According to the projection coordinates of the prediction frame, determine the projection height distance of the target person represented by the prediction frame;
[0040] The actual length of the imaging portion of the target person on the camera is calculated based on the projected body length distance and the actual distance between adjacent target grid points.
[0041] Optionally, the calculating the relative coordinates of the target person relative to the camera according to the actual distance value, the actual horizontal angle and the actual vertical angle specifically includes:
[0042] According to the actual distance value and the actual horizontal angle, the x-axis relative coordinate and the y-axis relative coordinate are calculated respectively;
[0043] Calculate the relative coordinate of the z-axis according to the actual distance value and the actual vertical angle;
[0044] According to the relative coordinates of the x-axis, the y-axis and the z-axis, the relative coordinates of the target person relative to the camera are obtained.
[0045] In the second aspect, the present application provides a personnel positioning system based on image analysis, which adopts the following technical solutions:
[0046] A personnel positioning system based on image analysis, comprising:
[0047] A data collection module is used to obtain the actual height of a person and an original image, and to construct a training set based on the actual height of the person and the image of the person; the original image is an image of a person taken by a camera;
[0048] A training module is used to train a preset target detection model using a training set to obtain a prediction model;
[0049] A prediction module, used for acquiring an image of a person to be located, inputting the image of the person to be located into the prediction model, and marking a prediction frame in the image of the person to be located by using the prediction model; wherein the image of the person to be located contains preset known grid points; and the prediction frame is used for marking a target person in the image of the person to be located;
[0050] An information acquisition module, used to acquire the internal parameters and location information of the camera; wherein the internal parameters include the size, resolution and focal length of the camera sensor;
[0051] An actual angle generation module, used to obtain the actual horizontal angle and the actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal parameters and the location information;
[0052] The personnel coordinate generation module is used to analyze the relative position of the target person and the camera according to the actual horizontal angle, the actual vertical angle, the internal parameters, the location information, the predicted frame of the image of the person to be located, and the known grid points to generate the actual coordinates of the target person.
[0053] In a third aspect, the present application provides a computer device, which adopts the following technical solution:
[0054] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program according to any one of the above methods.
[0055] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:
[0056] A computer-readable storage medium includes a computer program that can be loaded by a processor and executed in any of the above methods.
[0057] In summary, this application includes the following beneficial technical effects:
[0058] First, the original images containing people of different heights are collected, and combined with the actual height information of the people, a training data set is constructed, so that the preset target detection model can be trained with this data set to obtain a prediction model, so that the prediction model can generate a prediction frame that indicates the position of the target person in the image and reflects its relative size when the person is identified. Then, the actual horizontal angle and actual vertical angle of the target person relative to the camera are calculated by combining the imaging coordinates of the prediction frame in the image through the intrinsic parameters of the camera (including the physical size of the sensor, the resolution of the image, and the focal length of the camera) and the position information of the camera in the real world. Finally, the actual horizontal angle and actual vertical angle of the target person relative to the camera are calculated by combining the intrinsic parameters, position information, position of the prediction frame, and known grid points of the camera. The three-dimensional coordinates of the target person in the real world are not used to locate the target person. In the positioning process, there is no need to use cameras with positioning functions such as binocular cameras or depth of field cameras. Only ordinary cameras can be used, thereby reducing the cost of realizing personnel positioning through cameras. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 This is a flowchart of a personnel positioning method based on image analysis in one embodiment of the present application.
[0060] Figure 2 It is a schematic diagram of a known grid point setting in one embodiment of the present application.
[0061] Figure 3It is a flowchart of a method for constructing a training set according to one embodiment of the present application.
[0062] Figure 4 This is a flow chart of a method for generating an actual horizontal angle and an actual vertical angle in one embodiment of the present application.
[0063] Figure 5 This is a flow chart of a method for calculating the actual coordinates of a target person according to one embodiment of the present application.
[0064] Figure 6 This is a flow chart of a method for calculating the actual length of the imaging portion of a target person on a camera in one embodiment of the present application.
[0065] Figure 7 It is a schematic diagram of target grid point selection in one embodiment of the present application.
[0066] Figure 8 It is a schematic diagram of a prediction frame in perspective transformation coordinates according to one embodiment of the present application.
[0067] Fig. 9 It is a schematic diagram of the actual length of the imaging portion of a target person on a camera according to one embodiment of the present application.
[0068] Fig.10 It is a block diagram of a personnel positioning system based on image analysis according to one embodiment of the present application. DETAILED DESCRIPTION
[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0070] The present application embodiment discloses a personnel positioning method based on image analysis. Figure 1 , 2 , a personnel positioning method based on image analysis, comprising:
[0071] Step S101: obtaining the actual height of the person and the original image, and constructing a training set based on the actual height of the person and the image of the person;
[0072] The original image is an image of a person captured by a camera. When collecting the original image, the original image can be collected according to the commonly used shooting angle of the camera. For example, the camera is usually set at a higher position than the person, so the camera shoots the person from a bird's-eye view. When collecting the original image, the same bird's-eye view angle should also be used to collect the original image.
[0073] Step S102: Use the training set to train the preset target detection model to obtain a prediction model.
[0074] Among them, the preset target detection model can be any one of the YOLO (You Only Look Once) model, the SSD (Single Shot Multibox Detector) model and the DETR (Data-efficient Image Transformer) model.
[0075] Step S103: obtaining an image of the person to be located, and inputting the image of the person to be located into the prediction model, and marking a prediction box in the image of the person to be located by the prediction model;
[0076] Among them, the prediction box is used to mark the target person in the image of the person to be located.
[0077] Combination Figure 2 As an example, the image of the person to be located contains preset known grid points; the known grid points can be points located at the top corners of a chessboard, where the side length of the chessboard is The known grid points need to be set in advance within the shooting range of the camera, so that the positions of the known grid points in the real world are fixed, and the relative position relationship with the camera is also known, that is, the coordinates of the known grid points in the world coordinate system and the camera imaging coordinate system are known.
[0078] Step S104: Obtain the internal parameters and location information of the camera.
[0079] Among them, the internal parameters include the size, resolution and focal length of the camera sensor;
[0080] Step S105: obtaining the actual horizontal angle and the actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal parameters and the location information;
[0081] Step S106: Analyze the relative position of the target person and the camera according to the actual horizontal angle, the actual vertical angle, the internal parameters, the location information, the predicted frame of the image of the person to be located, and the known grid points to generate the actual coordinates of the target person.
[0082] In the above implementation, the original images containing people of different heights are first collected, and combined with the actual height information of the people, a training data set is constructed, so that the preset target detection model is trained using this data set to obtain a prediction model, so that the prediction model can generate a prediction frame that indicates the position of the target person in the image and reflects its relative size when the person is identified. Then, the actual horizontal angle and actual vertical angle of the target person relative to the camera are calculated by combining the imaging coordinates of the prediction frame in the image through the internal parameters of the camera (including the physical size of the sensor, the resolution of the image, and the focal length of the camera) and the position information of the camera in the real world. Finally, the actual horizontal angle and actual vertical angle of the target person relative to the camera are calculated by combining the internal parameters, position information, position of the prediction frame, and known grid points of the camera. The three-dimensional coordinates of the target person in the real world are not calculated, thereby realizing the positioning of the target person. In the positioning process, there is no need to use a camera with positioning function such as a binocular camera or a depth of field camera, and only an ordinary camera can be used, thereby reducing the cost of realizing personnel positioning through the camera.
[0083] Reference Figure 3 As an implementation of step S101, step S101 specifically includes:
[0084] Step S1011: Calculate the ratio between the actual height of the person and the preset standard height.
[0085] Among them, the preset standard height can be set to 1.7 meters.
[0086] Step S1012: scaling the initial target frame according to the ratio value to obtain a scaled target frame of the person.
[0087] The initial target frame is a frame with the person as the boundary, that is, the top of the initial target frame is close to the head of the person, and the bottom of the initial target frame is close to the feet of the person.
[0088] Specifically, when the initial target frame is scaled according to the proportional value, the bottom frame edge of the initial target frame is kept unchanged, and the top frame edge is moved up and down so that it proportionally reflects the preset standard height to obtain the scaled target frame. For example, if the actual height of the person is higher than the preset standard height, the top frame edge of the initial target frame is moved downward according to the proportional value, so that the height of the obtained scaled target frame only represents the height of the preset standard height. On the contrary, if the actual height of the person is lower than the preset standard height, the top frame edge of the initial target frame is moved upward according to the proportional value, so that the scaled target frame always represents the preset standard height. The annotation of the scaled target frame helps to enhance the accuracy and consistency of the target detection model training.
[0089] Step S1013: marking the zoom target frame in the original image, and calibrating the type label of each zoom target frame to obtain a sample image;
[0090] It should be understood that the type label is used to characterize the type of target within the zoom target box. In the present embodiment, since it is necessary to locate the personnel, the type label only includes the personnel type. The type label is calibrated so that only the targets of the personnel type are subsequently processed, which helps to reduce the computational complexity of the model.
[0091] Step S1014: constructing a training set based on the sample images.
[0092] As an implementation of step S102, step S102 specifically includes:
[0093] Step S1021: input each sample image in the training set into the target detection model, and output a priori boxes through the target detection model;
[0094] Step S1022: matching the prior frame and the scaled target frame to obtain a matching result, and calculating a positioning loss function of the target detection model according to the matching result;
[0095] As an example, the matching result may be an intersection-over-union (IOU) ratio between the prior frame and the scaled target frame. The larger the IOU ratio, the higher the matching degree between the prior frame and the scaled target frame.
[0096] Step S1023: iteratively update the model parameters of the target detection model according to the positioning loss function to obtain the prediction model.
[0097] Among them, the positioning loss function can be mean square error (MSE), smooth L1 loss (Smooth L1 Loss), etc. The positioning loss function calculates the coordinate difference between the predicted box and the real box (such as center coordinates, width, height, etc.), and converts these differences into loss values.
[0098] It should be understood that the scaled target box is a box that has been scaled proportionally. The prediction model is obtained by training the target detection model using sample images marked with the scaled target box, so that the prediction box finally output by the prediction model can maintain a specific height (i.e., the preset standard height), so that in the subsequent process of using the prediction box to calculate the position of the target person, even considering the prediction error of the prediction model, the height of the prediction box for adults can be better represented within the range of 1.6 to 1.8 meters, thereby reducing the final positioning error.
[0099] In the above implementation, the labeled training set images are input into the target detection model to output the priori boxes. These priori boxes are then matched with the labeled scaled target boxes to determine the priori boxes that are closest to the scaled target boxes. The positioning loss function is calculated based on the matching results to measure the difference between the target position predicted by the model and the scaled target box. Finally, the positioning loss function is minimized by iteratively updating the parameters of the model, thereby improving the accuracy of the model's prediction of the target position.
[0100] Reference Figure 4 As an implementation of step S105, step S105 specifically includes:
[0101] Step S1051: Generate the horizontal field of view angle and vertical field of view angle of the camera according to the size and focal length of the sensor;
[0102] Specifically, the horizontal field of view is: ; The vertical field of view is: ;in, and is the size of the sensor (width and height), and f is the focal length of the camera.
[0103] Step S1052: generating an imaging center point of the prediction box according to the imaging coordinates of the prediction box;
[0104] Specifically, the imaging center point of the prediction box is: ; Where (x, y) is the coordinate of the upper left corner of the prediction box; w is the width of the prediction box; h is the height of the prediction box; ( , ) is the imaging center point of the prediction box.
[0105] Step S1053: obtaining a first horizontal offset angle and a first vertical offset angle between the prediction frame and the camera center point according to the imaging center point, the horizontal field of view angle, the vertical field of view angle, and the resolution;
[0106] Specifically, the formula for calculating the first horizontal offset angle and the first vertical offset angle is: ;in, is the first horizontal offset angle; is the first vertical offset angle; W and H are the width and height of the camera resolution, respectively.
[0107] Step S1054: Acquire a second horizontal offset angle and a second vertical offset angle between the camera and the world coordinate system;
[0108] Specifically, the second horizontal offset angle and the second vertical offset angle between the camera and the world coordinate system can be obtained from the camera's PTZ (Pan-Tilt-Zoom). PTZ is a technology used for surveillance cameras that allows the camera to operate in three dimensions: Pan (horizontal rotation), Tilt (vertical rotation), and Zoom (zoom). Therefore, the horizontal angle θ (Pan) and the vertical angle θ (Zoom) can be obtained through the camera's PTZ. (Tilt), the horizontal angle θ represents the angle of rotation of the camera in the horizontal direction relative to its initial position. For example, the camera can be used as the origin, with the front of the camera as 0 degrees, and the angle of rotation to the left or right. Vertical angle Indicates the angle of rotation of the camera in the vertical direction relative to its initial position. For example, it can be the angle of rotation upward or downward with the camera as the origin and the horizontal angle of view of the camera as 0 degrees. In this embodiment, the horizontal angle θ is obtained as the second horizontal offset angle, and the vertical angle as the second vertical offset angle.
[0109] Step S1055: obtaining the actual horizontal angle of the target person according to the first horizontal offset angle and the second horizontal offset angle;
[0110] Step S1056: Obtain the actual vertical angle of the target person according to the first vertical offset angle and the second vertical offset angle.
[0111] Specifically, the actual horizontal angle and actual vertical angle calculation formula are: ;in, is the actual horizontal angle; is the actual vertical angle; is the second horizontal offset angle; is the second vertical offset angle.
[0112] In the above implementation, the first horizontal offset angle and the first vertical offset angle of the prediction frame relative to the center point of the camera are calculated to measure the degree of offset of the prediction frame relative to the center point of the camera. The second horizontal offset angle and the second vertical offset angle between the camera and the world coordinate system are then calculated to measure the degree of offset of the center point of the camera relative to the world coordinate system. Finally, the actual horizontal angle and the actual vertical angle of the target person in the world coordinate system can be obtained by combining the degree of offset of the prediction frame relative to the center point of the camera and the degree of offset of the center point of the camera relative to the world coordinate system.
[0113] Reference Figure 5 As an implementation of step S106, step S106 specifically includes:
[0114] Step S1061: According to the coordinates of the known grid points in the image of the person to be located, the prediction box is projected and transformed to generate the projection coordinates of the prediction box, and the actual length of the imaging part of the target person on the camera is obtained by analyzing the projection coordinates.
[0115] Step S1062: Generate the actual distance value between the target person and the camera according to the actual length and the internal parameters of the camera;
[0116] Specifically, according to the lens focal length imaging principle, the actual distance between the target person and the camera is: ; where f represents the focal length of the camera, h Indicates the height of the prediction box; It is the actual length of the target person's imaging portion on the camera.
[0117] Step S1063: Calculate the relative coordinates of the target person relative to the camera according to the actual distance value, the actual horizontal angle and the actual vertical angle;
[0118] Step S1064: superimpose the relative coordinates and the location information of the camera to obtain the actual coordinates of the target person.
[0119] Specifically, the actual coordinates of the target person are ;in,( X c , Y c , Z c ) is the location information of the camera; ( , , ) is the relative coordinate of the target person relative to the camera.
[0120] Reference Figure 6 As an implementation of step S1061, step S1061 specifically includes:
[0121] Step S10611: obtaining a number of known grid points around the target person in the image to be located as target grid points;
[0122] Combination Figure 7 , Figure 7 The schematic diagram of selecting the target grid point is illustrated. In this embodiment, since the known grid points are points located at the vertices of the chessboard, the number of target grid points can be four, that is, the four known grid points closest to the target person. Figure 7 Points A, B, C and D in the figure are the target grid points corresponding to the target personnel.
[0123] Step S10612: Obtain the perspective change coordinates and imaging coordinates of each target grid point.
[0124] It should be understood that the imaging coordinates are the coordinates in the camera imaging coordinate system. The perspective change coordinates can be preset according to actual needs. For example, in order to simplify the calculation, point A can be used as the origin in the perspective change coordinate system, and the distance between points AB and AC can be used as the unit length in the perspective change coordinate system; the direction of point B relative to point A is the horizontal axis direction, and the direction of point C relative to point A is the vertical axis direction. At this time, the perspective change coordinates of points A, B, C and D are (0, 0), (1, 0), (0, 1) and (1, 1) respectively.
[0125] Step S10613: Generate a perspective change matrix of the camera according to the perspective change coordinates and imaging coordinates of a plurality of target grid points.
[0126] Specifically, the perspective change matrix can be expressed as: ;in, is the element in the i-th row and j-th column of the perspective change matrix M.
[0127] The perspective transformation of each target grid point can be expressed as:
[0128] ; Wherein, (x′, y′) is the perspective change coordinate of the target grid point; (x, y) is the imaging coordinate of the target grid point.
[0129] From this, we can get: ;
[0130] ;
[0131] ;
[0132] Combined with homogeneous coordinates, x' and y' can be expressed as:
[0133] ;
[0134] ;
[0135] It should be understood that since there are eight unknowns in the perspective transformation matrix, by substituting the imaging coordinates of each set of points (A, B, C, D) and their corresponding perspective transformation coordinates {(0, 0), (1, 0), (0, 1), (1, 1)} into the relationship between x′ and y′, all elements in the perspective change matrix M can be solved, thereby realizing the generation of the perspective change matrix.
[0136] Step S10614: performing a projection transformation on the imaging coordinates of the prediction box according to the perspective change matrix to obtain the projection coordinates of the prediction box.
[0137] It should be understood that any point in the image of the person to be located can be converted into a perspective change coordinate system through the perspective change matrix. Therefore, the imaging coordinates of the prediction frame are projected and transformed through the perspective change matrix, so the projection coordinates of the prediction frame can be obtained.
[0138] Step S10615: determining the projected height distance of the target person represented by the prediction frame according to the projection coordinates of the prediction frame;
[0139] Combination Figure 8 , Figure 8 The image of the prediction box in the perspective change coordinate system is shown in the example. Based on the projection coordinates of the prediction box, the midpoint of the top edge of the prediction box is used as the head coordinate, and the midpoint of the bottom edge of the prediction box is used as the foot coordinate. The distance between the head coordinate and the foot coordinate is calculated to get the projected body length. Figure 8 middle is the head coordinate, is the foot coordinate, The projectile is a body length away.
[0140] Step S10616: Calculate the actual length of the imaging portion of the target person on the camera based on the projected body length and the actual distance between adjacent target grid points.
[0141] Specifically, the actual distance between adjacent target grid points is ; The scaling ratio of the world coordinate system and the perspective transformation coordinate system is obtained by the actual distance between adjacent target grid points in the world coordinate system and the distance between adjacent target grid points in the perspective transformation coordinate system. Then the projected body length is scaled by the scaling ratio to obtain the actual length of the imaging part of the target person on the camera.
[0142] Combination Fig. 9 , Fig. 9 The imaging part of the target person on the camera is illustrated. In this embodiment, the actual distance between two adjacent target grid points in the world coordinate system is , and the distance between two adjacent target grid points in the perspective change coordinate system is defined as a unit length of 1, so the scaling ratio of the world coordinate system and the perspective change coordinate system is ; The ratio between the actual length of the imaging part of the target person on the camera and the projected height of the target person in the perspective change coordinate system is also k. Therefore, the actual length of the imaging part of the target person on the camera is: .
[0143] As an implementation of step S1063, step S1063 specifically includes:
[0144] Step S10631: Calculate the x-axis relative coordinate and the y-axis relative coordinate according to the actual distance value and the actual horizontal angle;
[0145] Step S10632: Calculate the z-axis relative coordinate according to the actual distance value and the actual vertical angle;
[0146] Step S10633: Obtain the relative coordinates of the target person relative to the camera based on the x-axis relative coordinates, the y-axis relative coordinates, and the z-axis relative coordinates.
[0147] Specifically, the relative coordinates of the target person relative to the camera are:
[0148] ; Among them, ΔX is the relative coordinate of the x-axis; ΔY is the relative coordinate of the y-axis; ΔZ is the relative coordinate of the z-axis; D is the actual distance value; is the actual horizontal angle; is the actual vertical angle.
[0149] In summary, the technical principle of this embodiment is:
[0150] First, by recording the actual height of each person and adjusting it in proportion, it is ensured that all annotated boxes represent people of a specific height (such as 1.7 meters). The customized data annotation method improves the accuracy and consistency of the annotation, thereby improving the prediction box finally output by the prediction model to maintain a specific height. Secondly, this embodiment combines the target detection model and the principle of perspective geometry to accurately calculate the deviation angle of the target person relative to the center point of the camera and the distance of the target person relative to the camera. The accuracy of target positioning is increased. Then, not only the visual information of the target person in the image of the person to be located is combined, but also the actual physical size (assuming the preset standard height through the height) and the focal length of the camera are combined to provide a method for comprehensively calculating the distance and three-dimensional coordinates of the target person. Finally, compared with the technical solution that requires additional equipment (such as a depth of field camera and a binocular camera), the standard camera and image analysis technology in this embodiment can achieve the positioning of the target person, reduce the dependence on special hardware, reduce costs, and improve usability and ease of maintenance.
[0151] In addition, the embodiment of the present application discloses a personnel positioning system based on image analysis. The personnel positioning system based on image analysis can be applied to a computer device, which is a schematic diagram of the architecture of a computer device for implementing the above method provided by an embodiment of the present invention. In this embodiment, the computer device may include a personnel positioning system based on image analysis, a machine-readable storage medium, and a processor.
[0152] In this embodiment, the machine-readable storage medium and the processor may be located in the computer device and the two may be separately arranged. The machine-readable storage medium may also be independent of the computer device and accessed by the processor. The personnel positioning system based on image analysis may include multiple functional modules stored in the machine-readable storage medium, such as the various software functional modules included in the personnel positioning system based on image analysis. When the processor executes the computer program corresponding to the software functional module in the personnel positioning system based on image analysis, the personnel positioning system based on image analysis provided by the aforementioned method embodiment is implemented.
[0153] In this embodiment, the computer device may include one or more processors. The processor may process information and / or data related to the service request to perform one or more functions described in the present invention. In some embodiments, the processor may include one or more processing engines (e.g., a single-core processor or a multi-core processor). Just to give an example, the processor may include one or more hardware processors, such as a central processing unit CPU, an application-specific integrated circuit ASIC, a dedicated instruction set processor ASIP, a graphics processor GPU, a physical operation processing unit PPU, a digital signal processor DSP, a field programmable gate array FPGA, a programmable logic device PLD, a controller, a microcontroller unit, a reduced instruction set computer RISC, a microprocessor, etc., or similar or any combination thereof.
[0154] The machine-readable storage medium may store data and / or instructions. In some embodiments, the machine-readable storage medium may store acquired data or information. In some embodiments, the machine-readable storage medium may store data and / or instructions for execution or use by the computer device, and the computer device may implement the exemplary method described in the present application by executing or using the data and / or instructions. In some embodiments, the machine-readable storage medium may include a mass storage, a removable storage, a volatile read-write memory, a read-only memory ROM, etc., or any combination of the above examples. Exemplary mass storage may include a disk, an optical disk, a solid-state disk, etc. Exemplary removable storage may include a flash drive, a floppy disk, an optical disk, a memory card, a compressed disk, a magnetic tape, etc. Exemplary volatile read-write memory may include a random access memory RAM. Exemplary random access memory may include dynamic RAM, double rate synchronous dynamic RAM, static RAM, thyristor RAM, and zero capacitance RAM, etc. Exemplary ROM may include a masked ROM, a programmable ROM, an erasable programmable ROM, an electronically erasable programmable ROM, a compressed disk ROM, and a digital universal disk ROM, etc.
[0155] The personnel positioning system based on image analysis included in the computer device may include one or more software function modules. The software function modules may be programs and instructions stored in the machine-readable storage medium, and these software function modules are used to implement the above method when executed by the corresponding processor, for example, when executed by the processor of the drone, they are used to implement the method steps executed by the drone, or when executed by the computer device, they are used to implement the method steps executed by the computer device.
[0156] Refer to the detailed Fig.10 The embodiment of the present application discloses a personnel positioning system based on image analysis. A personnel positioning system based on image analysis includes:
[0157] A data collection module is used to obtain the actual height of a person and an original image, and to construct a training set based on the actual height of the person and the image of the person; the original image is an image of a person taken by a camera;
[0158] A training module is used to train a preset target detection model using a training set to obtain a prediction model;
[0159] A prediction module, used for acquiring an image of a person to be located, inputting the image of the person to be located into the prediction model, and marking a prediction frame in the image of the person to be located by using the prediction model; wherein the image of the person to be located contains preset known grid points; and the prediction frame is used for marking a target person in the image of the person to be located;
[0160] An information acquisition module, used to acquire the internal parameters and location information of the camera; wherein the internal parameters include the size, resolution and focal length of the camera sensor;
[0161] An actual angle generation module, used to obtain the actual horizontal angle and the actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal parameters and the location information;
[0162] The personnel coordinate generation module is used to analyze the relative position of the target person and the camera according to the actual horizontal angle, the actual vertical angle, the internal parameters, the location information, the predicted frame of the image of the person to be located, and the known grid points to generate the actual coordinates of the target person.
[0163] The personnel positioning system based on image analysis provided in the present application can implement the above-mentioned personnel positioning method based on image analysis, and the specific working process of the personnel positioning system based on image analysis can refer to the corresponding process in the above-mentioned method embodiment.
[0164] It should be noted that in the above embodiments, the description of each embodiment has different emphases, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0165] The present invention also discloses a computer-readable storage medium, which includes a computer program that can be loaded by a processor and executed as in any of the above methods.
[0166] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0167] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0168] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.
Claims
1. A personnel positioning method based on image analysis, characterized in that: include: Obtain the actual height of the person and the original image, and construct a training set based on the actual height of the person and the image of the person; The original image is an image containing a person taken by a camera; Use the training set to train the preset target detection model to obtain a prediction model; Acquire an image of a person to be located, input the image of the person to be located into the prediction model, and mark a prediction frame in the image of the person to be located by using the prediction model; wherein the image of the person to be located contains preset known grid points; the prediction frame is used to mark the target person in the image of the person to be located; Obtaining the internal parameters and location information of the camera; wherein the internal parameters include the size, resolution and focal length of the camera sensor; Obtaining an actual horizontal angle and an actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal reference and the location information; Analyze the relative position of the target person and the camera according to the actual horizontal angle, the actual vertical angle, the internal parameter, the location information, the predicted frame of the image of the person to be located, and the known grid points to generate the actual coordinates of the target person; The relative position of the target person and the camera is analyzed according to the actual horizontal angle, the actual vertical angle, the internal parameter, the location information, the predicted frame of the image of the person to be located, and the known grid points to generate the actual coordinates of the target person, specifically including: According to the coordinates of the known grid points in the image of the person to be located, the prediction frame is projected and transformed to generate the projection coordinates of the prediction frame, and the actual length of the imaging part of the target person on the camera is obtained by analyzing the projection coordinates; according to the actual length and the internal parameters of the camera, the actual distance value between the target person and the camera is generated; according to the actual distance value, the actual horizontal angle and the actual vertical angle, the relative coordinates of the target person relative to the camera are calculated; the relative coordinates and the location information of the camera are superimposed to obtain the actual coordinates of the target person.
2. The personnel positioning method based on image analysis according to claim 1, characterized in that: The obtaining of the actual height of the person and the original image, and constructing a training set according to the actual height of the person and the image of the person, specifically includes: Calculating the ratio between the actual height of the person and the preset standard height; Scaling the initial target frame according to the ratio value to obtain a scaled target frame of the person; Annotating the zoom target frame in the original image and calibrating the type label of each zoom target frame to obtain a sample image; A training set is constructed based on a number of the sample images.
3. The personnel positioning method based on image analysis according to claim 2 is characterized in that: The method of using the training set to train the preset target detection model to obtain the prediction model specifically includes: Input each sample image in the training set into the target detection model, and output the prior frame through the target detection model; Match the prior frame and the scaled target frame to obtain the matching result, and calculate the positioning loss function of the target detection model based on the matching result; The model parameters of the target detection model are iteratively updated according to the positioning loss function to obtain the prediction model.
4. The personnel positioning method based on image analysis according to claim 1, characterized in that: The obtaining of the actual horizontal angle and the actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal reference and the location information specifically includes: Generate a horizontal field of view angle and a vertical field of view angle of the camera according to the size of the sensor and the focal length; Generate an imaging center point of the prediction box according to the imaging coordinates of the prediction box; Obtaining a first horizontal offset angle and a first vertical offset angle between the prediction frame and the camera center point according to the imaging center point, the horizontal field of view angle, the vertical field of view angle, and the resolution; Get the second horizontal offset angle and the second vertical offset angle between the camera and the world coordinate system; Obtaining an actual horizontal angle of the target person according to the first horizontal offset angle and the second horizontal offset angle; The actual vertical angle of the target person is obtained according to the first vertical offset angle and the second vertical offset angle.
5. The personnel positioning method based on image analysis according to claim 1, characterized in that: The method of performing a projection transformation on the prediction frame according to the coordinates of the known grid points in the image of the person to be located to generate the projection coordinates of the prediction frame, and obtaining the actual length of the imaging portion of the target person on the camera by analyzing the projection coordinates, specifically includes: Acquire several known grid points around the target person in the image to be located as target grid points; Obtain the perspective change coordinates and imaging coordinates of each target grid point; Generate a perspective change matrix of the camera according to the perspective change coordinates and imaging coordinates of a plurality of the target grid points; According to the perspective change matrix, the imaging coordinates of the prediction box are projected and transformed to obtain the projection coordinates of the prediction box; According to the projection coordinates of the prediction frame, determine the projection height distance of the target person represented by the prediction frame; The actual length of the imaging portion of the target person on the camera is calculated based on the projected body length distance and the actual distance between adjacent target grid points.
6. The personnel positioning method based on image analysis according to claim 1, characterized in that: The calculation of the relative coordinates of the target person relative to the camera according to the actual distance value, the actual horizontal angle and the actual vertical angle specifically includes: According to the actual distance value and the actual horizontal angle, the x-axis relative coordinate and the y-axis relative coordinate are calculated respectively; Calculate the relative coordinate of the z-axis according to the actual distance value and the actual vertical angle; According to the relative coordinates of the x-axis, the y-axis and the z-axis, the relative coordinates of the target person relative to the camera are obtained.
7. A personnel positioning system based on image analysis, characterized in that: include: A data collection module is used to obtain the actual height of the person and the original image, and to construct a training set based on the actual height of the person and the image of the person; The original image is an image containing a person taken by a camera; A training module is used to train a preset target detection model using a training set to obtain a prediction model; A prediction module, used for acquiring an image of a person to be located, inputting the image of the person to be located into the prediction model, and marking a prediction frame in the image of the person to be located by using the prediction model; wherein the image of the person to be located contains preset known grid points; and the prediction frame is used for marking a target person in the image of the person to be located; An information acquisition module, used to acquire the internal parameters and location information of the camera; wherein the internal parameters include the size, resolution and focal length of the camera sensor; An actual angle generation module, used to obtain the actual horizontal angle and the actual vertical angle of the target person according to the imaging coordinates of the prediction frame, the internal parameters and the location information; The personnel coordinate generation module is used to analyze the relative position of the target person and the camera according to the actual horizontal angle, the actual vertical angle, the internal parameters, the location information, the predicted frame of the image of the person to be located, and the known grid points, so as to generate the actual coordinates of the target person; specifically, it is used to: according to the coordinates of the known grid points in the image of the person to be located, perform a projection transformation on the predicted frame to generate the projection coordinates of the predicted frame, and obtain the actual length of the imaging part of the target person on the camera by analyzing the projection coordinates; according to the actual length and the internal parameters of the camera, generate the actual distance value between the target person and the camera; according to the actual distance value, the actual horizontal angle and the actual vertical angle, calculate the relative coordinates of the target person relative to the camera; superimpose the relative coordinates and the location information of the camera to obtain the actual coordinates of the target person.
8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program according to any one of the methods of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The method comprises storing a computer program which can be loaded by a processor and execute the method according to any one of claims 1 to 6.
