Target detection complementing and positioning method and system based on personnel visual proportion

Abstract

The invention discloses a target detection complementing and positioning method and system based on a personnel visual proportion, and the method comprises the steps: collecting a whole-body image of a person in a scene, carrying out the cutting processing according to a corresponding visual proportion, and generating a training data set; based on the single-stage target detection model, performing model training by utilizing the training data set and the comprehensive loss function to obtain a visual proportion detection model with different visual proportion category identification capability; performing preliminary detection on personnel in the scene based on a preset target detection model, and cutting a personnel image area according to a detection result; taking the cut personnel image area as the input of a visual proportion detection model, and identifying a corresponding personnel visual proportion; and based on the identified personnel visual proportion and the corresponding detection frame pixel height, adjusting the detection frame to the position of the complete human body proportion, and calculating and outputting the actual spatial position of the corresponding personnel through a camera calibration method. According to the method, the personnel detection accuracy and the positioning precision in the shielded environment are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and processing, and in particular to a target detection, completion and positioning method and system based on human visual ratio. Background Art

[0002] Object detection technology is widely used in fields such as intelligent security, autonomous driving, and industrial production. In personnel management scenarios, accurate personnel detection and positioning are key to efficient management. For example, in crowded places like shopping malls, factories, and schools, accurate personnel location information can optimize personnel scheduling and ensure safe and orderly operation. Personnel trajectory analysis helps uncover patterns in personnel movements, providing data support for commercial layout adjustments and security planning.

[0003] However, in real-world monitoring scenarios, pedestrians are often obscured, making it difficult to obtain complete and accurate information about them. Traditional detection methods have limitations when dealing with obscured pedestrians, making it difficult to effectively detect and complete the obscured parts, making it impossible to effectively locate pedestrians.

[0004] Therefore, the existing technology needs to be improved. Summary of the Invention

[0005] The technical problem to be solved by the present invention is that, in response to the defects of the existing technology, the present invention provides a target detection, completion and positioning method and system based on the visible ratio of people, so as to solve the problem that traditional detection methods are difficult to effectively detect and complete the obscured parts and cannot effectively locate pedestrians when dealing with obscured pedestrians.

[0006] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0007] In a first aspect, the present invention provides a method for target detection, completion, and positioning based on a person's visual ratio, comprising:

[0008] Based on the visual ratio classification labels of people, full-body images of people in the scene are collected and cropped according to the corresponding visual ratio to generate a training dataset;

[0009] Based on the single-stage object detection model, the model is trained using the training dataset and the comprehensive loss function to obtain a visual scale detection model with different visual scale category recognition capabilities;

[0010] Perform preliminary detection of people in the scene based on the preset target detection model, and crop the image area of ​​the person according to the detection results;

[0011] Using the cropped person image region as input to the visual scale detection model to identify the corresponding person visual scale;

[0012] Based on the visual proportions of the identified person and the corresponding detection frame pixel height, the detection frame is adjusted to the position of the complete human body proportions, and the actual spatial position of the corresponding person is calculated and output through the camera calibration method.

[0013] In one implementation, the process of collecting full-body images of people in a scene and cropping them according to the corresponding visual ratios to generate a training dataset includes:

[0014] Collect full-body images of all people in the current scene and determine the minimum visible ratio based on the height of the head and the whole body in the image;

[0015] Determine the category of the visual ratio by multiplying the visual ratio by a preset multiple based on the visual ratio;

[0016] Cropping the full-body image according to the category of the visual ratio and generating a corresponding annotation file;

[0017] The number of the training data sets is determined according to the number of collected images and the number of categories, and a corresponding number of annotation files are selected to obtain the training data sets.

[0018] In one implementation, the performing model training using the training data set and the comprehensive loss function includes:

[0019] The category loss weight parameter in the comprehensive loss function is adjusted to enhance the recognition ability of the visual scale detection model for different categories of targets; wherein, the comprehensive loss function L total The formula is expressed as:

[0020] L total =λ box L box +λ obj L obj +λ cls L cls ;

[0021] The comprehensive loss function L total By the positioning loss L box , confidence loss L box And the category loss L cls Weighted, λ box ,λ obj ,λ cls They are positioning loss weight, confidence loss weight and category loss weight respectively;

[0022] The class loss L clsIt is used to measure the difference between the category probability distribution predicted by the model and the true category label, using the multi-classification cross entropy loss function, which is expressed as:

[0023]

[0024] Where C is the number of categories, y i is the true label of the i-th category, The probability of the i-th category predicted by the model.

[0025] In one implementation, performing preliminary detection of people in the scene based on a preset target detection model and cropping the image area of ​​the people according to the detection results includes:

[0026] Inputting the collected full-body images frame by frame into the preset target detection model to detect and classify the targets in the images;

[0027] Using the detection results of the preset target detection model, filter out detection boxes that belong to the corresponding person category and have a confidence level higher than a preset threshold;

[0028] Based on the coordinate information of the filtered detection frame, the boundary of the corresponding detection frame is determined, and the original scene person image is cropped to obtain the cropped person image area.

[0029] In one implementation, the preset target detection model includes: an R-CNN series model, a YOLO series model, an SSD series model, or a Transformer model.

[0030] In one implementation, adjusting the detection frame to a position corresponding to the proportions of a complete human body based on the recognized visible proportions of the person and the corresponding pixel height of the detection frame includes:

[0031] Based on the visual ratio of the identified person and the corresponding detection frame pixel height, the bottom position of the complete person detection frame is calculated:

[0032]

[0033] Where (X all ,Y all ) is to restore the pixel coordinates of the bottom of the detection frame; (X bottom ,Y bottom ) is the pixel coordinate of the bottom of the detection frame; where:

[0034]

[0035] Where (X top ,Y top ) is the pixel coordinate of the top of the detection box; r is the visual scale category being detected;

[0036] The position of the detection frame when the person is not blocked is obtained according to the calculated bottom position completion, so as to adjust the detection frame to a position with a complete human body proportion.

[0037] In one implementation, the calculating and outputting the actual spatial position of the corresponding person using a camera calibration method includes:

[0038] Collect image data information of calibration objects with known spatial coordinates at different angles and positions, obtain the internal and external parameters of the camera through analysis and processing of the calibration images, and construct a pixel-spatial coordinate mapping relationship model;

[0039] The pixel coordinate information of the detection frame after being adjusted to the complete human body proportions is extracted, and the corresponding spatial position of the person is calculated by combining the pixel-space coordinate mapping relationship model.

[0040] In a second aspect, the present invention provides a target detection, completion, and positioning system based on a person's visual ratio, comprising:

[0041] The data preprocessing module is used to classify labels based on the visual scale of people, collect full-body images of people in the scene, and crop them according to the corresponding visual scale to generate a training data set;

[0042] A model training module is used to perform model training based on a single-stage object detection model using the training data set and a comprehensive loss function to obtain a visual scale detection model with different visual scale category recognition capabilities;

[0043] A preliminary detection module is used to perform preliminary detection of people in the scene based on a preset target detection model and crop the image area of ​​the person according to the detection results;

[0044] A person visual ratio detection module is used to use the cropped person image area as input to the visual ratio detection model to identify the corresponding person visual ratio;

[0045] The completion and positioning module is used to adjust the detection frame to the position of the complete human body proportions based on the visual proportions of the identified person and the corresponding detection frame pixel height, and to infer and output the actual spatial position of the corresponding person through the camera calibration method.

[0046] In a third aspect, the present invention provides a terminal comprising: a processor and a memory, wherein the memory stores a target detection, completion and positioning program based on the visible ratio of a person, and the target detection, completion and positioning program based on the visible ratio of a person is used to implement the operation of the target detection, completion and positioning method based on the visible ratio of a person as described in the first aspect when executed by the processor.

[0047] In a fourth aspect, the present invention also provides a medium, which is a computer-readable storage medium, and which stores a target detection, completion and positioning program based on the visible ratio of a person. When the target detection, completion and positioning program based on the visible ratio of a person is executed by a processor, it is used to implement the operation of the target detection, completion and positioning method based on the visible ratio of a person as described in the first aspect.

[0048] The present invention adopts the above technical solution to achieve the following effects:

[0049] In the dataset annotation and preprocessing stage, the present invention introduces a person visual scale classification label and crops the images of people in the scene according to the visual scale, thereby generating a dataset suitable for training; in the model selection and training stage, a single-stage target detection model is adopted, and the loss weight of the recognition classification is increased during the training process to enhance the model's recognition ability for different visual scale categories; a general target detection model is used to perform preliminary detection of people in the scene, and the person image area is cropped according to the detection results as the input of the visual scale recognition model to achieve accurate recognition of the visual scale of the person in the input image; based on the identified person visual scale and the corresponding detection frame pixel height, the detection frame is adjusted to the position of the theoretical complete human body scale; finally, the actual spatial position of the person is calculated by combining the pixel-space coordinate mapping relationship obtained by camera calibration. The implementation of the present invention can significantly improve the accuracy of person detection and positioning accuracy in occluded environments, and has significant application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0051] Figure 1 This is a flow chart of the target detection, completion and positioning method based on the visible ratio of a person in the present invention.

[0052] Figure 2 Schematic diagram of an example of a detection model training data set in the present invention.

[0053] Figure 3 This is a schematic diagram of a complete person detection frame based on the visible ratio of the person in the present invention.

[0054] Figure 4 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0055] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0057] Exemplary Methods

[0058] Object detection technology is widely used in fields such as intelligent security, autonomous driving, and industrial production. In personnel management scenarios, accurate personnel detection and positioning are key to efficient management. For example, in places with high traffic, such as large shopping malls, factories, and schools, accurate personnel location information can optimize personnel scheduling and ensure safe and orderly operation. Personnel trajectory analysis helps uncover patterns in personnel movements, providing data support for commercial layout adjustments and security planning.

[0059] In intelligent monitoring and management systems, target detection technology plays an important role in personnel detection and positioning, personnel management, and trajectory analysis. However, pedestrians are often obscured in actual scenes, making it difficult to obtain personnel information. In addition, detection technology relies on fixed feature patterns and has limitations. When pedestrians are obscured, the lack of key features will cause information misjudgment, and existing technologies are difficult to effectively fill in the obscured parts. It is difficult to obtain high accuracy for direct personnel detection and positioning, and it is difficult to directly use it in applications such as scene personnel management and trajectory analysis.

[0060] In response to the above technical problems, an embodiment of the present invention provides a target detection, completion and positioning method based on the visible ratio of personnel. This method introduces visible ratio classification labels, optimizes the training process, accurately identifies the visible ratio of obscured pedestrians, completes detection information and accurately calculates the actual position, solves the occlusion problem, provides accurate data for personnel management to optimize scheduling, provides a reliable basis for trajectory analysis, and helps to upgrade application scenarios to intelligent levels. It is of great significance and has wide application value.

[0061] Specifically, first, in the dataset annotation and preprocessing stage, a person visual scale classification label is introduced, and full-body images of people in the scene are collected and cropped according to the visual scale to generate a training dataset; then, a single-stage target detection model is used for training, and the recognition classification loss weight is increased to enhance the recognition ability of different visual scale categories, thereby obtaining a visual scale detection model; then, a general target detection model is used to perform preliminary detection of people in the scene, and the image area of ​​the people is cropped based on the results, and it is input into the visual scale recognition model to accurately identify the visual scale of the people; then, based on the identified visual scale and the pixel height of the detection frame, the detection frame is adjusted to the theoretical complete human body scale position; finally, the pixel-space coordinate mapping relationship obtained by camera calibration is combined to obtain the positioning information of the people. Therefore, this embodiment can significantly improve the accuracy of person detection and positioning accuracy in occluded environments, and has significant application prospects.

[0062] like Figure 1 As shown, an embodiment of the present invention provides a method for target detection, completion, and positioning based on a person's visible ratio, comprising the following steps:

[0063] Step S100 : According to the visual ratio classification labels of the persons, full-body images of the persons in the scene are collected, and the images are cropped according to the corresponding visual ratios to generate a training data set.

[0064] In this embodiment, during the dataset annotation and preprocessing stage, by introducing a person visual scale classification label, full-body images of people in the scene are collected and cropped according to the visual scale to generate a dataset suitable for training.

[0065] Specifically, in one implementation of this embodiment, step S100 includes the following steps:

[0066] Step S101: capturing full-body images of all persons in the current scene, and determining the minimum visible ratio based on the height of the head and the whole body in the image;

[0067] Step S102, multiplying the visual ratio by a preset multiple to determine the category of the visual ratio;

[0068] Step S103, cropping the full-body image according to the category of the visual ratio, and generating a corresponding annotation file;

[0069] Step S104 , determining the number of the training data set according to the number of collected images and the number of categories, and selecting a corresponding number of annotation files to obtain the training data set.

[0070] In this embodiment, first, full-body image data of a person in a monitoring scene is collected, and the head and body of the person are labeled in the person image data using the LabelImg annotation tool, and the annotation information is saved; then, the labeled data are used to calculate the ratio of the pixel height of the head and the pixel height of the whole body of each person using Python and perform a statistical average, and the integer ratio closest to the average is used as the minimum visible ratio category benchmark; and based on this minimum visible ratio benchmark, the category of the visible ratio is determined by multiplying by an integer multiple (for example, 1.5 times, 2 times, 2.5 times, etc.); wherein, when determining the category of the visible ratio, the category of the visible ratio is determined when the minimum visible ratio is multiplied by the corresponding multiple and does not exceed 100%; finally, based on the determination of all visible categories, all collected person image data are cropped using Python, and corresponding annotation files are generated to construct a visual ratio model training set.

[0071] As an example, in this embodiment, in a specific application scenario, the training dataset for the visual scale detection model is generated by a labeling file that conforms to the YOLOv5 dataset labeling format and the corresponding image data. The labeling file contains a txt file with five key elements: the target class in the image, the center coordinates of the target box (x, y), and the width and height (w, h) of the target box.

[0072] Furthermore, obtaining the model training data set requires collecting scene personnel data and preprocessing it, specifically including:

[0073] 1) Select a background and take full-body images of the on-site personnel from different angles as the initial image data;

[0074] 2) Based on the image data, the head and body of the person in the image are labeled using the LabelImg annotation tool;

[0075] 3) Use Python scripts to batch process these annotated images, calculate the pixel ratio of the head to the whole body in each image, average these ratios and round them to the nearest integer as the minimum visible ratio benchmark;

[0076] 4) Based on the minimum visible ratio determined above, the base ratio is expanded by integer multiples to divide the image into multiple categories;

[0077] 5) Based on these divided visual ratio categories, the collected full-body image of the person is cropped according to the visual ratio and a corresponding annotation configuration file that conforms to the YOLO annotation format is generated based on the cropped image to generate a data set suitable for training. The example of the detection model training data set in this embodiment is as follows: Figure 2 shown.

[0078] like Figure 1 As shown, an embodiment of the present invention provides a method for target detection, completion, and positioning based on a person's visible ratio, comprising the following steps:

[0079] Step S200 , based on the single-stage object detection model, the model is trained using the training data set and the comprehensive loss function to obtain a visual scale detection model with different visual scale category recognition capabilities.

[0080] In this embodiment, a single-stage target detection model is adopted, and the loss weight of the recognition classification is increased during the training process to improve the model's recognition ability for different visual scale categories, and a visual scale detection model is trained.

[0081] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0082] Step S201 : adjusting the category loss weight parameters in the comprehensive loss function to enhance the recognition capability of the visual scale detection model for targets of different categories.

[0083] In this embodiment, the loss weight of the model recognition classification is increased during the training process to improve the model's ability to identify different visual scale categories, and a visual scale detection model is obtained through training; specifically:

[0084] Define the comprehensive loss function L total , the formula is:

[0085] L total =λ box L box +λ obj L obj +λ cls L cls

[0086] Where the positioning loss L box , confidence loss L box , category loss L cls Weighted, λ box ,λ obj ,λ cls They are positioning loss weight, confidence loss weight and category loss weight respectively;

[0087] Class loss L cls It is used to measure the difference between the category probability distribution predicted by the model and the true category label, using the multi-classification cross entropy loss function, which is expressed as:

[0088]

[0089] Where C is the number of categories, y iis the true label of the i-th category, The probability of the i-th category predicted by the model;

[0090] In this embodiment, the category weight ratios of the three losses are adjusted, and the weight of the category loss is particularly increased, thereby improving the model's ability to recognize categories with different visual proportions.

[0091] like Figure 1 As shown, an embodiment of the present invention provides a method for target detection, completion, and positioning based on a person's visible ratio, comprising the following steps:

[0092] Step S300: Perform preliminary detection of people in the scene based on a preset target detection model, and crop the image area of ​​the people according to the detection results.

[0093] In this embodiment, a general target detection model is used to perform preliminary detection on people in the scene, and objects belonging to the person category are screened out and the image area is cropped.

[0094] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0095] Step S301: input the collected full-body image frame by frame into the preset target detection model to detect and classify the targets in the image;

[0096] Step S302: Using the detection results of the preset target detection model, filter out detection boxes that belong to the corresponding person category and have a confidence level higher than a preset threshold;

[0097] Step S303 : Based on the coordinate information of the filtered detection frame, the boundary of the corresponding detection frame is determined, and the original scene person image is cropped to obtain the cropped person image area.

[0098] In this embodiment, the target category, image position range and corresponding confidence level in the video image are detected frame by frame.

[0099] Specifically, a target detection model is selected that has excellent performance and is widely used in the field of target detection and can output the target category and target image position range (detection frame). The target detection model (that is, the preset target detection model) can be an R-CNN series model, a YOLO series model, an SSD series model or a Transformer model, etc.

[0100] Based on the detection results output by the general model, detection frames belonging to the person category and with a confidence level higher than a preset threshold (for example, a confidence level higher than 90%) are screened out to obtain their detection frame information; then, based on the coordinate information of the screened person detection frame and following the boundary of the detection frame, the original scene person image is cropped to obtain the person area image.

[0101] like Figure 1 As shown, an embodiment of the present invention provides a method for target detection, completion, and positioning based on a person's visible ratio, comprising the following steps:

[0102] Step S400, using the cropped person image region as input to the visual scale detection model to identify the corresponding person visual scale;

[0103] In step S500 , based on the visual proportions of the identified person and the corresponding detection frame pixel height, the detection frame is adjusted to a position that matches the proportions of the entire human body, and the actual spatial position of the corresponding person is calculated and output through a camera calibration method.

[0104] In this embodiment, the obtained person area image is further used as input, and the proportion of the photographic part of the person in the image to the whole body is obtained through visual proportion model detection, so as to achieve accurate recognition of the visual proportion of the person in the input image; and, based on the identified person visual proportion and the corresponding detection frame pixel height, the detection frame is adjusted to the position of the theoretical complete human body proportion, and finally, combined with the pixel-space coordinate mapping relationship obtained through camera calibration, the actual spatial position of the person is calculated.

[0105] Specifically, in one implementation of this embodiment, step S500 includes the following steps:

[0106] Step S501: Based on the identified visible ratio of the person and the corresponding detection frame pixel height, the bottom position of the detection frame of the complete person is calculated:

[0107]

[0108] Where (X all ,Y all ) is to restore the pixel coordinates of the bottom of the detection frame; (X bottom ,Y bottom ) is the pixel coordinate of the bottom of the detection frame; where:

[0109]

[0110] Where (X top ,Y top ) is the pixel coordinate of the top of the detection box; r is the visual scale category being detected;

[0111] Step S502 , completing the position of the detection frame when the person is not blocked according to the calculated bottom position, so as to adjust the detection frame to a position with a complete human body proportion.

[0112] In this embodiment, according to the visual ratio information of the person, the theoretical position information of the detection frame when the pedestrian is not blocked is obtained; first, a dictionary is constructed according to the visual ratio category and the actual visual ratio value, and then according to the visual ratio category of the person's detection frame and the visual ratio value corresponding to the image height of the detection frame, the position of the detection frame is restored, that is, the position of the detection frame with the theoretical complete human body proportion is completed; in this embodiment, the complete person detection frame is completed based on the visual ratio of the person, such as Figure 3 shown.

[0113] Specifically, in one implementation of this embodiment, step S500 includes the following steps:

[0114] Step S503: collecting image data information of a calibration object with known spatial coordinates at different angles and positions, obtaining the intrinsic and extrinsic parameters of the camera by analyzing and processing the calibration image, and constructing a pixel-spatial coordinate mapping relationship model;

[0115] Step S504 , extracting pixel coordinate information of the detection frame after being adjusted to the proportion of the complete human body, and calculating the corresponding spatial position of the person by combining the pixel-space coordinate mapping relationship model.

[0116] In this embodiment, a pixel-space coordinate mapping relationship model is constructed based on the relationship between the actual position of the scene space calibration object and the image coordinates. Specifically, image data information of the calibration object with known spatial coordinates at different angles and positions is collected to obtain a calibration reference point dataset. The spatial coordinate calibration object data can be constructed by recording the spatial points of artificially recorded chessboard calibration plates, dot calibration plates, natural scene objects, etc. and the coordinate information of their projection points in the image to construct the calibration reference point data.

[0117] The obtained calibration reference point dataset is used to solve the intrinsic and extrinsic parameters of the camera based on the imaging projection relationship. The algorithm for solving the camera parameters can be through the cv2.calibrateCamera() function in the OpenCV library in Python, the estimateCameraParameters() function in the computer vision toolbox of Matlab, or other fitting algorithms.

[0118] Based on the solved camera parameters, a pixel-space coordinate mapping relationship model is constructed. According to this mapping relationship, the spatial coordinates corresponding to the entities in the image are inferred, and the spatial coordinate information corresponding to all pixels of the image entity is obtained; finally, the corresponding spatial position of the person is inferred based on the image position of the complete detection frame of the person and the entity pixel-space pixel coordinate information.

[0119] To sum up, this embodiment addresses the problem that it is difficult to accurately locate a person when he is obscured during target detection and positioning. Based on the demand for personnel detection and positioning, self-built personnel images of different visual scales are used to construct a visual scale training data set and optimize the training model parameters to obtain a personnel visual scale detection model; first, a general target detection model is used to perform a preliminary detection on the image, and the target classified as a person is screened out. The personnel image data is cropped and input into the visual scale model to obtain its visual scale, and the personnel image position when not obscured is obtained based on the visual scale; then the pixel-space coordinate data corresponding to the scene calibration object is collected, and the camera parameters are solved through the camera calibration program to construct the image entity pixel-space coordinate mapping relationship information, and the personnel image position obtained through the completion is combined to infer the position of the person in the actual space, which can effectively deal with target occlusion and achieve high-precision personnel detection and positioning, and can provide effective data reference for scene applications such as personnel management, early warning, and trajectory analysis.

[0120] This embodiment achieves the following technical effects through the above technical solution:

[0121] In the dataset annotation and preprocessing stage, this embodiment introduces a person visual scale classification label and crops the images of people in the scene according to the visual scale, thereby generating a dataset suitable for training. In the model selection and training stage, a single-stage target detection model is adopted, and the loss weight of the recognition classification is increased during the training process to enhance the model's recognition ability for different visual scale categories. A general target detection model is used to perform preliminary detection of people in the scene, and the person image area is cropped based on the detection results as the input of the visual scale recognition model to achieve accurate recognition of the visual scale of the people in the input image. Based on the identified person visual scale and the corresponding detection frame pixel height, the detection frame is adjusted to the position of the theoretical complete human body scale. Finally, the actual spatial position of the person is calculated by combining the pixel-space coordinate mapping relationship obtained through camera calibration. This embodiment can significantly improve the accuracy of person detection and positioning accuracy in occluded environments, and has significant application prospects.

[0122] Exemplary devices

[0123] Based on the above embodiment, the present invention further provides a target detection, completion, and positioning system based on a person's visual ratio, comprising:

[0124] The data preprocessing module is used to classify labels based on the visual scale of people, collect full-body images of people in the scene, and crop them according to the corresponding visual scale to generate a training data set;

[0125] A model training module is used to perform model training based on a single-stage object detection model using the training data set and a comprehensive loss function to obtain a visual scale detection model with different visual scale category recognition capabilities;

[0126] A preliminary detection module is used to perform preliminary detection of people in the scene based on a preset target detection model and crop the image area of ​​the person according to the detection results;

[0127] A person visual ratio detection module is used to use the cropped person image area as input to the visual ratio detection model to identify the corresponding person visual ratio;

[0128] The completion and positioning module is used to adjust the detection frame to the position of the complete human body proportions based on the visual proportions of the identified person and the corresponding detection frame pixel height, and to infer and output the actual spatial position of the corresponding person through the camera calibration method.

[0129] This embodiment achieves the following technical effects through the above technical solution:

[0130] In the dataset annotation and preprocessing stage, this embodiment introduces a person visual scale classification label and crops the images of people in the scene according to the visual scale, thereby generating a dataset suitable for training. In the model selection and training stage, a single-stage target detection model is adopted, and the loss weight of the recognition classification is increased during the training process to enhance the model's recognition ability for different visual scale categories. A general target detection model is used to perform preliminary detection of people in the scene, and the person image area is cropped based on the detection results as the input of the visual scale recognition model to achieve accurate recognition of the visual scale of the people in the input image. Based on the identified person visual scale and the corresponding detection frame pixel height, the detection frame is adjusted to the position of the theoretical complete human body scale. Finally, the actual spatial position of the person is calculated by combining the pixel-space coordinate mapping relationship obtained through camera calibration. This embodiment can significantly improve the accuracy of person detection and positioning accuracy in occluded environments, and has significant application prospects.

[0131] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 4 shown.

[0132] The terminal includes: a processor, memory, interface, display screen and communication module connected via a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or other devices.

[0133] When the computer program is executed by a processor, it is used to implement the operation of the target detection, completion and positioning method based on the visual ratio of a person.

[0134] It will be understood by those skilled in the art that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0135] In one embodiment, a terminal is provided, which includes: a processor and a memory, wherein the memory stores a target detection, completion and positioning program based on the visible ratio of a person, and the target detection, completion and positioning program based on the visible ratio of a person is used to implement the operation of the target detection, completion and positioning method based on the visible ratio of a person when executed by the processor.

[0136] In one embodiment, a storage medium is provided, wherein the storage medium stores a target detection, completion, and positioning program based on a person's visible ratio. When the target detection, completion, and positioning program based on a person's visible ratio is executed by a processor, it is used to implement the operations of the above-mentioned target detection, completion, and positioning method based on a person's visible ratio.

[0137] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include both non-volatile and volatile memory.

[0138] In summary, the present invention provides a method and system for target detection, completion and positioning based on the visual proportion of people, including: collecting full-body images of people in the scene, and cropping them according to the corresponding visual proportion to generate a training data set; based on a single-stage target detection model, using the training data set and a comprehensive loss function to train the model to obtain a visual proportion detection model with different visual proportion category recognition capabilities; based on a preset target detection model, preliminary detection of people in the scene is performed, and the image area of ​​the people is cropped according to the detection results; the cropped image area of ​​the people is used as the input of the visual proportion detection model to identify the corresponding visual proportion of the people; based on the identified visual proportion of the people and the corresponding pixel height of the detection frame, the detection frame is adjusted to the position of the complete human body proportion, and the actual spatial position of the corresponding person is calculated and output through a camera calibration method. The present invention significantly improves the accuracy of people detection and positioning accuracy in occluded environments.

[0139] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for target detection, completion and positioning based on human visual ratio, characterized in that: include: Based on the visual ratio classification labels of people, full-body images of people in the scene are collected and cropped according to the corresponding visual ratio to generate a training dataset; Based on the single-stage object detection model, the model is trained using the training dataset and the comprehensive loss function to obtain a visual scale detection model with different visual scale category recognition capabilities; Perform preliminary detection of people in the scene based on the preset target detection model, and crop the image area of ​​the person according to the detection results; Using the cropped person image region as input to the visual scale detection model to identify the corresponding person visual scale; Based on the visual proportions of the identified person and the corresponding detection frame pixel height, the detection frame is adjusted to the position of the complete human body proportions, and the actual spatial position of the corresponding person is calculated and output through the camera calibration method.

2. The target detection, completion and positioning method based on human visual ratio according to claim 1, characterized in that: The method of collecting full-body images of people in the scene based on the classification labels of the people's visual proportions and cropping them according to the corresponding visual proportions to generate a training data set includes: Collect full-body images of all people in the current scene and determine the minimum visible ratio based on the height of the head and body in the image; Multiplying the visual ratio by a preset multiple based on the visual ratio to determine the category of the visual ratio; Cropping the full-body image according to the category of the visual ratio and generating a corresponding annotation file; The number of the training data sets is determined according to the number of collected images and the number of categories, and a corresponding number of annotation files are selected to obtain the training data sets.

3. The method for target detection, completion and positioning based on human visual ratio according to claim 1, characterized in that: The method of performing model training using the training data set and the comprehensive loss function includes: The category loss weight parameter in the comprehensive loss function is adjusted to enhance the recognition ability of the visual scale detection model for different categories of targets; wherein, the comprehensive loss function L total The formula is expressed as: L total =λ box L box +λ obj L obj +λ cls L cls ; The comprehensive loss function L total By the positioning loss L box , confidence loss L box And the category loss L cls Weighted, λ box ,λ obj ,λ cls They are positioning loss weight, confidence loss weight and category loss weight respectively; The class loss L cls It is used to measure the difference between the category probability distribution predicted by the model and the true category label, using the multi-classification cross entropy loss function, which is expressed as: Where C is the number of categories, y i is the true label of the i-th category, The probability of the i-th category predicted by the model.

4. The method for target detection, completion and positioning based on human visual ratio according to claim 1, characterized in that: The preliminary detection of people in the scene based on the preset target detection model and cropping of the person image area according to the detection results include: Inputting the collected full-body images frame by frame into the preset target detection model to detect and classify the targets in the images; Using the detection results of the preset target detection model, filter out detection boxes that belong to the corresponding person category and have a confidence level higher than a preset threshold; Based on the coordinate information of the filtered detection frame, the boundary of the corresponding detection frame is determined, and the original scene person image is cropped to obtain the cropped person image area.

5. The method for target detection, completion and positioning based on human visual ratio according to claim 4, characterized in that: The preset target detection model includes: R-CNN series model, YOLO series model, SSD series model or Transformer model.

6. The method for target detection, completion and positioning based on human visual ratio according to claim 1, characterized in that: The step of adjusting the detection frame to a position proportional to the entire human body based on the visual proportion of the identified person and the corresponding pixel height of the detection frame includes: Based on the visual ratio of the identified person and the corresponding detection frame pixel height, the bottom position of the complete person detection frame is calculated: Where (X all ,Y all ) is to restore the pixel coordinates of the bottom of the detection frame; (X bottom ,Y bottom ) is the pixel coordinate of the bottom of the detection frame; where: Where (X top ,Y top ) is the pixel coordinate of the top of the detection box; r is the visual scale category being detected; The position of the detection frame when the person is not blocked is obtained according to the calculated bottom position completion, so as to adjust the detection frame to a position with a complete human body proportion.

7. The method for target detection, completion and positioning based on human visual ratio according to claim 1, characterized in that: The method of calculating and outputting the actual spatial position of the corresponding person by using the camera calibration method includes: Collect image data information of calibration objects with known spatial coordinates at different angles and positions, obtain the internal and external parameters of the camera through analysis and processing of the calibration images, and construct a pixel-spatial coordinate mapping relationship model; The pixel coordinate information of the detection frame after being adjusted to the complete human body proportions is extracted, and the corresponding spatial position of the person is calculated by combining the pixel-space coordinate mapping relationship model.

8. A target detection, completion and positioning system based on human visual ratio, characterized in that: include: The data preprocessing module is used to classify labels based on the visual scale of people, collect full-body images of people in the scene, and crop them according to the corresponding visual scale to generate a training data set; A model training module is used to perform model training based on a single-stage object detection model using the training data set and a comprehensive loss function to obtain a visual scale detection model with different visual scale category recognition capabilities; A preliminary detection module is used to perform preliminary detection of people in the scene based on a preset target detection model and crop the image area of ​​the person according to the detection results; A person visual ratio detection module is used to use the cropped person image area as input to the visual ratio detection model to identify the corresponding person visual ratio; The completion and positioning module is used to adjust the detection frame to the position of the complete human body proportions based on the visual proportions of the identified person and the corresponding detection frame pixel height, and to infer and output the actual spatial position of the corresponding person through the camera calibration method.

9. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a target detection, completion, and positioning program based on a person's visible ratio, and when the target detection, completion, and positioning program based on a person's visible ratio is executed by the processor, it is used to implement the operation of the target detection, completion, and positioning method based on a person's visible ratio as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a target detection, completion, and positioning program based on a person's visible ratio. When the target detection, completion, and positioning program based on a person's visible ratio is executed by a processor, it is used to implement the operation of the target detection, completion, and positioning method based on a person's visible ratio as described in any one of claims 1 to 7.

Images