Method and device for determining the direction of an object relative to itself

Abstract

This specification provides a method and device for determining the direction of an object moving relative to itself, wherein the method for determining the direction of an object moving relative to itself includes: obtaining an image to be identified; identifying a target object in the image to be identified, and obtaining target detection frame information and target detection feature point information of the target object; generating target tracking feature point information of the target object in the image to be identified based on the target detection frame information and the target detection feature point information; generating an orientation feature vector and a displacement feature vector of the target object based on the target tracking feature point information; and determining the target moving direction of the target object relative to itself based on the orientation feature vector and the displacement feature vector. The determination of the target moving direction of the target object relative to itself is achieved, and the accuracy of determining the target moving direction is improved.

Description

Technical Field

[0001] The present invention relates to the field of rail transportation technology, and in particular to a method for determining a direction of an object relative to itself. The present invention also relates to a device for determining a direction of an object relative to itself, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In current practical applications, more and more places are beginning to deploy gates to restrict pedestrian traffic, such as subway stations, high-speed rail stations, office buildings, etc. Pedestrians need to hold the corresponding gate card or key to open the gate and pass through the gate. This process is generally achieved by judging a series of infrared counter-radiation sensors installed on the inside of the gate to assist diffuse reflection sensors. It can sense the entry and exit status of people and control the opening and closing of the gate at the appropriate time, but it cannot accurately identify the direction of travel of people. If the direction of travel of people cannot be accurately judged, the opening and closing of the gate may cause accidental injuries to people. Therefore, how to accurately judge the direction of travel of pedestrians in the gate is crucial. Summary of the invention

[0003] In view of this, an embodiment of this specification provides a method for determining the direction of an object relative to itself. One or more embodiments of this specification also relate to a device for determining the direction of an object relative to itself, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a method for determining a moving direction of an object relative to itself is provided, comprising:

[0005] Obtain an image to be recognized;

[0006] Identify a target object in the image to be identified, and obtain target detection frame information and target detection feature point information of the target object;

[0007] Generating target tracking feature point information of the target object in the image to be identified according to the target detection frame information and the target detection feature point information;

[0008] Generating an orientation feature vector and a displacement feature vector of the target object according to the target tracking feature point information;

[0009] The target moving direction of the target object relative to itself is determined according to the orientation feature vector and the displacement feature vector.

[0010] According to a second aspect of the embodiments of this specification, there is provided a device for determining a moving direction of an object relative to itself, comprising:

[0011] An acquisition module is configured to acquire an image to be recognized;

[0012] A recognition module is configured to recognize a target object in the image to be recognized, and obtain target detection frame information and target detection feature point information of the target object;

[0013] A first generating module is configured to generate target tracking feature point information of the target object in the image to be identified according to the target detection frame information and the target detection feature point information;

[0014] A second generating module is configured to generate a direction feature vector and a displacement feature vector of the target object according to the target tracking feature point information;

[0015] The determination module is configured to determine the target moving direction of the target object relative to itself according to the orientation feature vector and the displacement feature vector.

[0016] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0017] Memory and processor;

[0018] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the method for determining the direction of movement of an object relative to itself are implemented.

[0019] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the method for determining the direction of travel of an object relative to itself are implemented.

[0020] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for determining the direction of travel of an object relative to itself.

[0021] An embodiment of the present specification realizes that, by performing target detection on an image to be identified, target detection frame information and target detection feature point information of a target object in the image to be identified are obtained, and target tracking feature point information of the target object is determined based on the target detection frame information and the target detection feature point information. Based on the target tracking feature point information, an orientation feature vector and a displacement feature vector are further generated for determining the target moving direction of the target object relative to itself, so that the target moving direction of the target object relative to itself is determined based on the orientation feature vector and the displacement feature vector of the target object, thereby improving the accuracy of determining the target moving direction of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of an application scenario of a method for determining the direction of an object relative to itself provided by an embodiment of this specification;

[0023] Figure 2 is a flow chart of a method for determining the direction of an object relative to itself provided by an embodiment of this specification;

[0024] Figure 3 is a schematic diagram of a target tracking frame and target tracking feature points provided by an embodiment of this specification;

[0025] Figure 4 is a flowchart of a processing process of a method for determining a moving direction of an object relative to itself provided by an embodiment of this specification;

[0026] Figure 5 It is a schematic diagram of the structure of a device for determining the direction of an object moving relative to itself provided by an embodiment of this specification;

[0027] Figure 6 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0028] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0029] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0030] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0031] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] First, the terms involved in one or more embodiments of this specification are explained.

[0033] Tracking algorithm: It is a technique used to track a specific target in a video sequence. It is widely used in the fields of computer vision and image processing. The main goal of a tracking algorithm is to identify and track one or more target objects in consecutive video frames. For example, SORT (SIMPLE ONLINE AND REALTIME TRACKING, a target tracking algorithm based on Kalman filtering) algorithm, DeepSORT (SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEPASSOCIATION METRIC, a target tracking algorithm based on deep learning) algorithm, BoT-SORT algorithm, ByteTrack (byte tracking algorithm) algorithm, etc.

[0034] YOLOv8n-pose model: a pre-trained pose estimation model.

[0035] In current practical applications, more and more places are beginning to deploy gates to restrict pedestrian traffic, such as subway stations, high-speed rail stations, office buildings, etc. Pedestrians need to hold the corresponding gate card or key to open the gate and pass through the gate. This process is generally achieved by judging the diffuse reflection sensor assisted by a series of infrared counter-radiation sensors installed on the inside of the gate, which can sense the entry and exit status of personnel and control the gate opening and closing actions at the appropriate time. However, in real life, some pedestrians will deliberately break through the gate. For example, in subway stations, some passengers deliberately break through the gate to evade tickets. For example, in office buildings, office workers may deliberately break through the gate when they are about to be late. These behaviors cannot be detected by infrared counter-radiation sensors in time, and it is impossible to determine whether there is a deliberate gate-breaking behavior when the pedestrian is passing. In addition, in the scenario of intelligent automatic doors, it is also necessary to determine the direction of travel of pedestrians, and control whether to open or close the intelligent automatic door according to the direction of travel of pedestrians. Therefore, in practical applications, it is essential to accurately judge the direction of travel of pedestrians.

[0036] In this specification, a method for determining the direction of an object traveling relative to itself is provided. This specification also relates to an apparatus for determining the direction of an object traveling relative to itself, a computing device, a computer-readable storage medium, and a computer program product, each of which is described in detail in the following embodiments.

[0037] See also Figure 1 , Figure 1 The following is a schematic diagram showing an application scenario of a method for determining the direction of an object relative to itself according to an embodiment of the present specification. Figure 1 As shown in the figure, the passenger is passing in the opposite direction of the subway gate, which means that the passenger has intentionally entered the gate in the opposite direction. However, in actual applications, the passenger's passing status is often identified by a series of infrared sensors on the inside of the subway gate, but they cannot be used to determine the passenger's direction of travel. Therefore, it is not possible to accurately determine whether the passenger has intentionally entered the gate based only on the infrared sensors.

[0038] Based on this, the method for determining the direction of travel of an object relative to itself provided in this specification can obtain multiple images of the subway gate based on an image acquisition device installed above the subway gate, and perform target detection based on the multiple images obtained to detect and identify the personnel in the image and determine the direction of travel of the personnel. Specifically, a tracking algorithm can be used to track the target person to determine the direction and trajectory of travel of the target person, so as to determine whether the target person has a gate-breaking behavior. In practical applications, an image acquisition device (such as a color camera, etc.) can be used to collect a video sequence of the subway gate, and subsequent processing can be performed based on multiple frames of images in the video sequence. Based on this, the subsequent processing process is briefly described by taking any two frames of images in the video sequence (hereinafter referred to as the current frame image and the previous frame image) as an example. After obtaining the current frame image, the previous frame image of the current frame image, and the reference detection frame information and reference tracking feature point information of the target person in the previous frame image, target detection is performed on the current frame image to identify the target detection frame information and target detection feature point information of the target person in the current frame image. Among them, both the tracking feature points and the detection feature points can include the center point of the human body, the back spine point, the chest center point, the left shoulder point, the right shoulder point, etc. The tracking feature points are specifically the feature points corresponding to the target person tracking frame, and the detection feature points are specifically the feature points corresponding to the target person detection frame. The tracking feature points and the detection feature points include but are not limited to the above five feature points, which can be determined based on the actual application situation, and this manual does not limit them here.

[0039] After obtaining the reference detection frame information of the previous frame image and the target detection frame information of the current frame image, a tracking algorithm (such as SORT algorithm, DeepSORT algorithm, etc.) can be used to calculate based on the reference detection frame information and the target detection frame information to generate the target tracking frame information of the target person in the current frame image. Then, according to the target detection frame information, the target tracking frame information and the target detection feature point information, the target tracking feature point information corresponding to the target tracking frame information is matched, so that the orientation feature vector and the displacement feature vector of the target person can be further generated based on the target tracking feature point information of the current frame image and the reference tracking feature point information of the previous frame image, and the target travel direction of the target person is determined by calculating the angle between the orientation feature vector and the displacement feature vector. The specific implementation method of determining the target travel direction of the target person based on the angle between the orientation feature vector and the displacement feature vector is described in detail in the following method for determining the direction of travel of an object relative to itself, and this manual will not elaborate on it here.

[0040] An embodiment provided in this specification realizes that, by performing target detection on the target person, the target detection frame information and target detection feature point information of the target person in the current frame image, as well as the reference detection frame information and reference detection feature point information of the target person in the previous frame image are generated, and the target tracking frame information of the target person in the current frame image is predicted using a tracking algorithm based on the target detection frame information and the reference detection frame information, and the target tracking feature point information corresponding to the target tracking frame information is determined. Thus, it is possible to respectively generate the orientation feature vector and the displacement feature vector of the target person based on the target tracking feature information of the current frame image and the reference tracking feature point information of the previous frame image, and determine the target travel direction of the target person in the subway gate based on the angle between the orientation feature vector and the displacement feature vector, thereby improving the accuracy of determining the target travel direction of the target person, and further improving the accuracy of identifying whether the person has intentionally broken through the gate.

[0041] See also Figure 2 , Figure 2 A flow chart of a method for determining the direction of an object relative to itself provided according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0042] Step 202: Obtain an image to be recognized.

[0043] In actual applications, more and more application scenarios need to determine the direction of travel or the body rotation angle of the person. For example, in the application scenarios of subway gates and access control, it can be judged whether the pedestrian has broken through the gate based on the direction of travel and the body rotation angle of the pedestrian; in the application scenarios of smart automatic doors, it can be judged whether the pedestrian passes through the smart automatic door and the direction of travel of the pedestrian based on the direction of travel and the body rotation angle of the pedestrian, so as to more accurately determine whether the smart automatic door needs to be opened or closed.

[0044] Among them, the image to be identified refers to the image or image frame of the device to be passed obtained based on the image acquisition device. The image acquisition device can specifically be a monocular structured light camera, a binocular structured light camera or other color camera. Preferably, in one or more embodiments provided in this specification, the image acquisition device adopts a top-down perspective, that is, the device lens faces the ground, and the physical center of the camera lens and the line connecting the lens focus are perpendicular to the ground. Based on this, the image to be identified acquired by the image acquisition device is also a color image from a top-down perspective. The device to be passed refers to the equipment that pedestrians need to pass, such as subway gates, access control gates, smart automatic doors, etc.

[0045] Step 204: Identify the target object in the image to be identified, and obtain target detection frame information and target detection feature point information of the target object.

[0046] After acquiring the image to be identified, it is necessary to determine the target detection frame information and target detection feature point information of the target object in the image to be identified, so that in the subsequent process, the target tracking feature point information of the target object in the image to be identified can be generated according to the target detection frame information and the target detection feature point information, so as to track the target object and determine the target moving direction.

[0047] Among them, the target object refers to any pedestrian passing through the device to be passed. The target detection frame information refers to the detection frame information of the target object in the image to be identified, and can specifically include the target detection frame identifier and the target detection frame position information. The target detection frame identifier specifically refers to the detection frame identifier of the target object in the image to be identified; the target detection frame position information specifically refers to the detection frame position information of the target object in the image to be identified. The target detection feature point information refers to the detection feature point information of the target object in the image to be identified, and the detection feature point information can be obtained by performing target detection on the image. The target detection feature point information can specifically include the target detection center feature point information, the target detection lateral feature point information and the target detection longitudinal feature point information. The central feature point of target detection can be specifically understood as the central feature point of the human body of the target object obtained based on target detection. The central feature point information of target detection refers to the position information of the central feature point of target detection; the lateral feature point of target detection includes the left shoulder feature point and the right shoulder feature point of the target object obtained based on target detection. The lateral feature point information of target detection specifically includes the left shoulder feature point information and the right shoulder feature point information. The left shoulder feature point information refers to the position information of the left shoulder feature point, and the right shoulder feature point information refers to the position information of the right shoulder feature point; the longitudinal feature point of target detection includes the chest feature point and the spine feature point of the target object obtained based on target detection. The longitudinal feature point information of target detection includes the chest feature point information and the spine feature point information. The chest feature point information is specifically the position information of the chest feature point, and the spine feature point is specifically the position information of the spine feature point.

[0048] Specifically, target detection may be performed on the image to be identified to identify the target object in the image to be identified, and to obtain target detection frame information and target detection feature point information of the target object in the image to be identified.

[0049] Based on this, in a specific implementation manner provided in this specification, identifying a target object in the image to be identified and obtaining target detection frame information and target detection feature point information of the target object include:

[0050] Inputting the image to be recognized into a target detection model;

[0051] Obtain target detection frame information and target detection feature point information output by the target detection model.

[0052] The target detection model is used to detect targets in images or image frames to output target detection frame information and target detection feature point information of the target object. In practical applications, the target detection model can be a YOLOv8n-pose model or a modified model of other models. For example, a modified model for detecting feature point prediction can be added on the basis of the yolov5 model.

[0053] Specifically, after obtaining the image to be identified, the image to be identified can be directly input into the target detection model to use the target detection model to perform target detection on the target object in the image to be identified, and obtain the target detection frame information and target detection feature point information of the target object in the image to be identified.

[0054] Since the target detection model is the basis for detecting targets in images, it is necessary to train the target detection model in advance before using it to detect targets in images, so as to ensure the accuracy of the target detection box information and target detection feature point information obtained by using the target detection model. The following explains the training process of the target detection model.

[0055] In a specific implementation provided in this specification, the target detection model is trained based on the following method:

[0056] Acquire sample training data, wherein the sample training data includes an image to be trained, sample detection frame information of the image to be trained, and sample detection feature point information;

[0057] Inputting the image to be trained into the target detection model to obtain predicted detection frame information and predicted detection feature point information of the image to be trained;

[0058] Calculate a first loss value according to the predicted detection frame information and the sample detection frame information, and calculate a second loss value according to the predicted detection feature point information and the sample detection feature point information;

[0059] According to the first loss value and the second loss value, the model parameters of the target detection model are adjusted, and the target detection model is continuously trained until a training stop condition is reached.

[0060] The sample training data refers to the sample data used to train the target detection model, and the sample training data includes the image to be trained, the sample detection frame information of the image to be trained, and the sample detection feature point information. The image to be trained is the image or image frame that the target detection model needs to perform target detection during the training of the target detection model; the sample detection frame information is the actual detection frame information corresponding to the image to be trained; and the sample detection feature point information is the actual detection feature point information corresponding to the image to be trained.

[0061] The sample training data may be sample data obtained in a sample training data set (including an image to be trained, sample detection frame information and sample detection feature point information), and the sample detection frame information and the sample detection feature point information may be obtained by data annotation; the sample training data set refers to a set consisting of an image to be trained, sample detection frame information and sample detection feature point information; predicted detection frame information refers to the detection frame information output by inputting the image to be trained into a target detection model; predicted detection feature point information refers to the detection feature point information output by inputting the image to be trained into a target detection model; the first loss value refers to the difference between the sample detection frame information and the predicted detection frame information, and is used to measure the difference between the sample detection frame information and the predicted detection frame information; the second loss value refers to the difference between the sample detection feature point information and the predicted detection feature point information, and is used to measure the difference between the sample detection feature point information and the predicted detection feature point information.

[0062] Specifically, the image to be trained, the sample detection frame information and the sample detection feature point information are obtained, and the image to be trained is input into the target detection model to obtain the predicted detection frame information and the predicted detection feature point information. At this time, the target detection model is a model that has not been trained yet. There will be deviations between the determined predicted detection frame information and the actual sample detection frame information, and between the predicted detection feature point information and the actual sample detection feature point information. The model parameters of the target detection model need to be adjusted accordingly. Specifically, the first loss value of the target detection model is calculated according to the output predicted detection frame information and the sample detection frame information, and the second loss value of the target detection model is calculated according to the output predicted detection feature point information and the sample detection feature point information. The loss function for calculating the model loss value can be a 0-1 loss function, a square loss function, a cross entropy loss function, etc. in actual applications. In this specification, preferably, a cross entropy loss function or a square loss function is selected as the loss function for calculating the model loss value, and the model parameters of the target detection model are adjusted according to the first loss value and the second loss value. The adjusted model parameters are used for the next batch of sample training data to continue training the target detection model until the stop condition of the model training is reached.

[0063] Specifically, the model training stopping conditions include that the model loss value is less than a preset threshold and / or the training rounds reach a preset round.

[0064] In a specific implementation provided in this specification, taking the model loss value being less than a preset threshold as an example of a training stop condition, the preset threshold is 0.2. When both the first loss value and the second loss value are less than 0.2, the target detection model training is considered completed.

[0065] In another specific implementation provided in the present specification, taking the preset number of training rounds as the training stop condition as an example, the preset number of training rounds is 30 rounds. When the number of training rounds of the training sample training data reaches 30 rounds, the target detection model training is considered to be completed.

[0066] In another specific implementation provided in the present specification, two training stop conditions, a preset threshold and a preset training round, are set, and the model loss value and the training round are determined at the same time. When either the model loss value or the training round meets the training stop condition, the target detection model training is considered to be completed.

[0067] In one embodiment provided in the present specification, target detection is performed on an image by training a target detection model to identify the target object present in the image, and output target detection frame information and target detection feature point information of the target object, so that in a subsequent process, the target object can be tracked based on the target detection frame information and the target detection feature point information, and the target tracking frame information and the target tracking feature point information of the target object can be further obtained.

[0068] Step 206: Generate target tracking feature point information of the target object in the image to be identified based on the target detection frame information and the target detection feature point information.

[0069] After obtaining the target detection frame information and target detection feature point information of the target object in the image to be identified, the target tracking feature point information of the target object in the image to be identified can be generated according to the target detection frame information and the target detection feature point information. The specific implementation method is as follows:

[0070] In a specific implementation provided in this specification, generating target tracking feature point information of the target object in the image to be identified according to the target detection frame information and the target detection feature point information includes:

[0071] Obtaining reference detection frame information of the target object in the reference recognition image;

[0072] Generating target tracking frame information of the target object according to the reference detection frame information and the target detection frame information;

[0073] The target tracking feature point information of the target object in the image to be identified is obtained according to the target detection frame information, the target tracking frame information and the target detection feature point information.

[0074] Among them, the reference recognition image refers to the previous image or the previous image frame of the image to be recognized in the time sequence. The reference detection frame information refers to the detection frame information of the target object in the reference recognition image, which may specifically include the reference detection frame identifier and the reference detection frame position information. The reference detection frame identifier specifically refers to the detection frame identifier of the target object in the reference recognition image; the reference detection frame position information refers to the detection frame position information of the target object in the reference recognition image. The target tracking frame information refers to the tracking frame information of the target object in the image to be recognized, which may specifically include the target tracking frame identifier and the target tracking frame position information. The target tracking frame identifier specifically refers to the tracking frame identifier of the target object in the image to be recognized.

[0075] Specifically, after obtaining the target detection frame information of the target object in the image to be identified, the tracking algorithm can be used to track the target object according to the reference detection frame information and the target detection frame information of the target object to generate the target tracking frame information of the target object. Thus, the target tracking feature point information of the target object in the image to be identified can be obtained according to the target detection frame information, the target tracking frame information and the target detection feature point information. Since the tracking frame is generated to achieve tracking of the target object, the target tracking frame identifier is also the target object identifier of the target object.

[0076] In a specific implementation provided in this specification, generating target tracking frame information of the target object according to the reference detection frame information and the target detection frame information includes:

[0077] Acquire reference detection frame position information from the reference detection frame information, and acquire target detection frame position information and a target detection frame identifier from the target detection frame information;

[0078] Calculating target tracking frame position information of the target object according to the reference detection frame position information, the target detection frame position information and the tracking algorithm;

[0079] Determine the target detection frame identifier as a target tracking frame identifier;

[0080] Based on the target tracking frame identifier and the target tracking frame position information, target tracking frame information of the target object is generated.

[0081] Specifically, the reference detection frame position information of the reference detection frame is obtained in the reference detection frame information, and the target detection frame position information and the target detection frame identifier of the target detection frame are obtained in the target detection frame information. The reference detection frame position information and the target detection frame position information are calculated using a tracking algorithm to obtain the target tracking frame position information of the target object. In practical applications, the tracking algorithm may specifically be a SORT algorithm, a DeepSORT algorithm, a ByteTrack, etc., and this specification does not limit the use of the tracking algorithm. The target tracking frame identifier is determined based on the target detection frame identifier. Specifically, the target detection frame identifier may be determined as the target tracking frame identifier, or the target tracking frame identifier of the target tracking frame may be customized after obtaining the target tracking frame position information, and then the target tracking frame identifier and the target tracking frame position information are determined as the target tracking frame information of the target object.

[0082] In one embodiment provided in the present specification, a tracking algorithm is used to track and predict the target object based on the reference detection frame information and the target detection frame information of the target object to obtain the target tracking frame information of the target object, thereby tracking the target object and determining the target moving direction of the target object.

[0083] In practical applications, the detection frame is available for the target object in different images or different image frames, and the tracking frame is used to track the movement of the target object in multiple images or multiple image frames. Although both the detection frame and the tracking frame can identify the target object, there may be deviations in the positions of the detection frame and the tracking frame for the same image or the same image frame, that is, the detection frame and the tracking frame cannot overlap, and there may be errors in directly using the target detection feature point information corresponding to the target detection frame to determine the target movement direction of the subsequent target object. Therefore, according to the target detection frame information and the target detection feature point information, the corresponding feature points can be matched for the target tracking frame, and the corresponding feature point information can be determined.

[0084] Based on this, in a specific implementation provided in this specification, obtaining target tracking feature point information of the target object in the image to be identified according to the target detection frame information, the target tracking frame information and the target detection feature point information includes:

[0085] Determining a first corresponding relationship between the target detection frame and the target tracking frame according to the target detection frame information and the target tracking frame information;

[0086] Determining a second corresponding relationship between the target detection frame and the target detection feature point according to the target detection frame information and the target detection feature point information;

[0087] According to the first corresponding relationship and the second corresponding relationship, the target tracking feature points corresponding to the target tracking frame are matched to obtain the target tracking feature point information of the target object in the image to be identified.

[0088] The first correspondence refers to the correspondence between the target detection frame and the target tracking frame, which can be determined based on the target detection frame position information and the target tracking frame position information. The second correspondence refers to the correspondence between the target detection frame and the target detection feature point, which can be determined based on the target detection frame position information and the target detection feature point position information.

[0089] Specifically, after determining the target tracking frame information of the target object according to the reference detection frame information and the target detection frame information, determine the first correspondence between the target detection frame and the target tracking frame according to the target detection frame position information of the target detection frame and the target tracking frame position information of the target tracking frame, and determine the second correspondence between the target detection frame and the target detection feature point according to the target detection frame position information of the target detection frame and the target detection feature point position information of the target detection feature point. According to the first correspondence and the second correspondence, match the tracking feature points corresponding to the target tracking frame, and determine the tracking feature point information of each tracking feature point. If there is a situation where the target detection frame and the target tracking frame overlap, the target detection feature point can be directly determined as the target tracking feature point, and the target detection feature point information is the target tracking feature point information.

[0090] Further, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a target tracking frame and target tracking feature points provided according to an embodiment of the present specification. The image to be identified is an image frame in a video sequence. Figure 3 As shown, the dotted box area is the image to be identified, and the upper left corner of the image to be identified is the origin of the coordinate system. The coordinate system can be a three-dimensional space coordinate system. The x-axis, y-axis, and z-axis of the coordinate system are perpendicular to each other, and the positive directions of the three axes satisfy the right-hand screw rule. The x-axis is the horizontal axis of the coordinate system, the y-axis is the vertical axis of the coordinate system, and the z-axis is the vertical axis of the coordinate system. The directions of the x-axis and y-axis are as follows: Figure 3 As shown in , the z-axis direction is perpendicular to the image to be identified and points inward. The pedestrian in the image to be identified is the target object, and the rectangular frame surrounding the target object is the target tracking frame. The area between the rectangular frame and the dotted frame is the rest of the video screen. Figure 3 Points A, B, C, D and E shown in the figure are specifically target tracking feature points. Specifically, point A is the center feature point of the human body of the target object, point B is the spine feature point of the target object, and point C is the chest (chest center point) feature point of the target object. Point D is the left shoulder feature point of the target object, and point E is the right shoulder feature point of the target object.

[0091] In one embodiment provided in the present specification, after performing target detection on the image to be identified and obtaining the target detection frame information and target detection feature point information of the target object, a first correspondence between the target detection frame and the target tracking frame, as well as a second correspondence between the target detection frame and each target detection feature point can be further determined. Based on the first correspondence and the second correspondence, the tracking feature points corresponding to the target tracking frame are matched, and the tracking feature point information of each tracking feature point is determined, thereby improving the accuracy of determining the target tracking feature point information.

[0092] Step 208: Generate an orientation feature vector and a displacement feature vector of the target object according to the target tracking feature point information.

[0093] In current practical applications, infrared sensors are usually used to detect whether there are people within their detection range. However, infrared sensors cannot be used to determine the direction of travel and the angle of rotation of the human body, and thus cannot accurately identify the direction of travel.

[0094] The method for determining the direction of an object relative to itself provided in this specification can determine the target direction of the target object based on the target tracking feature point information and the reference tracking feature point information after determining the target tracking feature point information of the target object. The specific implementation method is as follows:

[0095] In a specific implementation provided in this specification, generating the orientation feature vector and the displacement feature vector of the target object according to the target tracking feature point information includes:

[0096] Acquire reference tracking feature point information of the target object in the reference recognition image;

[0097] Generating a direction feature vector of the target object according to the target tracking feature point information;

[0098] A displacement feature vector of the target object is generated according to the target tracking feature point information and the reference tracking feature point information.

[0099] In practical applications, the target direction of travel of the target object can be determined by the orientation direction and movement direction of the target object. Based on this, after determining the target tracking feature point information of the target object, this manual combines the reference tracking feature point information of the target object in the previous image or the previous image frame to determine the orientation direction and movement direction of the target object, thereby determining the target direction of travel of the target object.

[0100] Among them, the reference tracking feature point information refers to the tracking feature point information of the target object in the reference recognition image, and the reference tracking feature point information can specifically include reference center feature point information, reference horizontal feature point information, and reference vertical feature point information. The reference center feature point can be specifically understood as the center feature point of the human body of the target object, and the reference center feature point information includes the position information of the reference center feature point; the reference horizontal feature point can specifically include the left shoulder feature point and the right shoulder feature point of the target object, and the reference horizontal feature point information includes the left shoulder feature point information and the right shoulder feature point information. The left shoulder feature point information is specifically the position information of the left shoulder feature point, and the right shoulder feature point information is specifically the position information of the right shoulder feature point. The reference vertical feature point can specifically include the chest feature point (specifically, the chest center feature point) and the spine feature point (back spine point) of the target object, and the reference vertical feature point information includes the chest feature point information and the spine feature point information. The chest feature point information is specifically the position information of the chest feature point, and the spine feature point is specifically the position information of the spine feature point. The orientation feature vector is used to represent the orientation direction of the target object, and is a vector expression of the orientation direction of the target object. The displacement feature vector is used to represent the moving direction of the target object, and is a vector expression of the moving direction of the target object.

[0101] Specifically, according to the target tracking feature point information obtained by matching (specifically, the position information of the target tracking feature point), the orientation feature vector of the target object is calculated and generated, and according to the target tracking feature point information and the reference tracking feature point information (specifically, the position information of the reference tracking feature point), the displacement feature vector of the target object is calculated and generated. Then, the target moving direction of the target object can be determined according to the orientation feature vector and the displacement feature vector of the target object.

[0102] The following explains how the orientation feature vector of the target object is generated:

[0103] In a specific implementation provided in this specification, generating the orientation feature vector of the target object according to the target tracking feature point information includes:

[0104] Acquire target horizontal feature point information and target vertical feature point information from the target tracking feature point information;

[0105] Generating a first orientation feature vector of the target object according to the target lateral feature point information;

[0106] Generating a second orientation feature vector of the target object according to the target longitudinal feature point information;

[0107] An orientation feature vector of the target object is generated according to the first orientation feature vector and the second orientation feature vector.

[0108] Among them, the first orientation feature vector refers to the feature vector generated by the target horizontal feature points, specifically the feature vector generated by the left shoulder point feature points and the right shoulder point feature points; the second orientation feature vector refers to the feature vector generated by the target longitudinal feature points, specifically the feature vector generated by the chest feature points and the spine feature points.

[0109] Specifically, the orientation feature vector of the target object can be determined using the target horizontal feature points of the target object, and the displacement feature vector can be determined using the target longitudinal feature points of the target object. Based on this, the target horizontal feature point information and the target longitudinal feature point information are obtained in the target tracking feature point information, and the first orientation feature vector of the target object is generated according to the target horizontal feature point information, and the second orientation feature vector of the target object is generated according to the target longitudinal feature point information, so as to generate the orientation feature vector of the target object according to the first orientation feature vector and the second orientation feature vector.

[0110] Furthermore, as described above, the first orientation feature vector is specifically generated by the left shoulder feature point and the right shoulder feature point of the target object. Therefore, the following further explains the process of generating the first orientation feature vector based on the left shoulder feature point and the right shoulder feature point.

[0111] In a specific implementation provided in this specification, generating a first orientation feature vector of the target object according to the target lateral feature point information includes:

[0112] Determining left shoulder point feature point information and right shoulder point feature point information in the target lateral feature point information;

[0113] Calculating a lateral feature vector according to the feature point information of the left shoulder point and the feature point information of the right shoulder point;

[0114] Unitizing the transverse eigenvector to obtain a transverse unit vector of the transverse eigenvector;

[0115] The lateral unit vector is rotated according to a preset rotation matrix to generate a first orientation feature vector of the target object.

[0116] Among them, the horizontal feature vector can be specifically understood as a directed vector with the left shoulder point feature point as the starting point and the right shoulder point feature point as the end point; the horizontal unit vector is the unit vector of the horizontal feature vector.

[0117] Specifically, the left shoulder feature point information and the right shoulder feature point information are determined in the target horizontal feature point information, and a horizontal feature vector with the left shoulder feature point as the starting point and the right shoulder feature point as the end point is calculated and generated according to the left shoulder feature point information and the right shoulder feature point information. The horizontal feature vector is normalized to obtain a unit vector of the horizontal feature vector, that is, a horizontal unit vector. The horizontal unit vector is rotated by a preset rotation matrix to obtain a rotated horizontal unit vector, that is, a first orientation feature vector.

[0118] Furthermore, based on the above Figure 3 Take the example for explanation, where the left shoulder feature point is point D, and the left shoulder feature point information (that is, the position information of the left shoulder feature point) is ( x D , y D ), the right shoulder feature point is point E, and the right shoulder feature point information (that is, the position information of the right shoulder feature point) is ( x E , y E ), according to the feature point information of the left shoulder point ( x D , y D ) and the right shoulder feature point information ( x E , y E ), calculate and generate the lateral eigenvector . The horizontal eigenvector Normalize to obtain the horizontal unit vector . According to the preset rotation matrix N , the horizontal unit vector Rotate 90 degrees counterclockwise to get the first orientation eigenvector , preset rotation matrix N Can be set to .

[0119] In one embodiment provided in this specification, during the process of calculating and generating the first orientation feature vector, the feature vector may be normalized to reduce the complexity and difficulty of calculating the first orientation feature vector and further ensure the accuracy of the calculated and generated first orientation feature vector.

[0120] Furthermore, as described above, the second orientation feature vector is specifically generated by the chest feature points and spine feature points of the target object. Therefore, the following further explains the process of generating the second orientation feature vector based on the chest feature points and spine feature points.

[0121] In a specific implementation provided in this specification, generating a second orientation feature vector of the target object according to the target longitudinal feature point information includes:

[0122] Determining chest feature point information and spine feature point information in the target longitudinal feature point information;

[0123] Calculating a longitudinal feature vector according to the chest feature point information and the spine feature point information;

[0124] The longitudinal feature vector is normalized to generate a second orientation feature vector of the target object.

[0125] The longitudinal feature vector can be specifically understood as a directed vector starting from the spine feature point and ending at the chest feature point.

[0126] Specifically, the chest feature point information and the spine feature point information are determined in the target longitudinal feature point information, and a longitudinal feature vector with the spine feature point as the starting point and the chest feature point as the end point is calculated and generated based on the chest feature point information and the spine feature point information. The longitudinal feature vector is normalized to obtain a normalized longitudinal unit vector, i.e., the second orientation feature vector.

[0127] Furthermore, based on the above Figure 3 Take the example for explanation, where the chest feature point is point C, and the chest feature point information (that is, the position information of the chest feature point) is ( x C , y C ), the spine feature point is point B, and the spine feature point information (that is, the position information of the spine feature point) is ( x B , y B ), based on the chest feature point information ( x C , y C ) and spine feature point information ( x B , y B ), calculate and generate the longitudinal eigenvector . The vertical feature vector Normalize to obtain the vertical unit vector , which is the second orientation eigenvector .

[0128] Furthermore, when calculating the first orientation feature vector and the second oriented eigenvector , the orientation feature vector of the target object can be calculated according to the following formula 1:

[0129] Formula 1

[0130] in, is the orientation feature vector of the target object.

[0131] In an embodiment provided in this specification, in the process of calculating and generating the second orientation feature vector, the feature vector is also normalized to reduce the complexity and difficulty of calculating the second orientation feature vector, and further ensure the accuracy of the calculated and generated second orientation feature vector.

[0132] The following is an explanation of how the displacement feature vector of the target object is generated:

[0133] In a specific implementation provided in this specification, generating a displacement feature vector of the target object according to the target tracking feature point information and the reference tracking feature point information includes:

[0134] Acquire target center feature point information from the target tracking feature point information, and acquire reference center feature point information from the reference tracking feature point information;

[0135] Calculating a candidate displacement feature vector according to the target center feature point information and the reference center feature point information;

[0136] The candidate displacement feature vectors are normalized to generate a displacement feature vector of the target object.

[0137] Among them, the displacement feature vector is specifically a unitized candidate displacement feature vector; the candidate displacement feature vector is specifically a feature vector generated by the target center feature point and the reference center feature point. Since the displacement feature vector is used to characterize the moving direction of the target object, the candidate movement vector can specifically be a feature vector with the reference center feature point as the starting point and the target center feature point as the end point.

[0138] Specifically, the target center feature point information is obtained from the target tracking feature point information, and the reference center feature point information is obtained from the reference tracking feature point information. According to the target center feature point information and the reference center feature point information, a candidate displacement feature vector is calculated and generated with the reference center feature point as the starting point and the target center feature point as the end point. The candidate displacement feature vector is normalized to generate a normalized candidate displacement feature vector, i.e., a displacement feature vector.

[0139] Further, continue with the above Figure 3Take this as an example to explain, where the target center feature point is point A, and the target center feature point information (that is, the location information of the target center feature point) is ( x A , y A ), the reference center feature point is point A' ( Figure 3 (not shown), the reference center feature point information (that is, the position information of the reference center feature point) is ( x A’ , y A’ ). According to the target center feature point information ( x A , y A ) and reference center feature point information ( x A’ , y A’ ), calculate and generate candidate displacement feature vectors , normalize the candidate displacement feature vector to obtain the displacement feature vector .

[0140] In one embodiment provided in the present specification, in the process of calculating and generating a displacement feature vector, the displacement feature vector of the target object is calculated based on the position information corresponding to the reference center feature point in the previous image and the target center feature point in the current image, and a method of unitizing the feature vector is adopted to reduce the complexity and difficulty of calculating the displacement feature vector and ensure the accuracy of the calculated displacement feature vector.

[0141] Step 210: Determine the target moving direction of the target object relative to itself according to the orientation feature vector and the displacement feature vector.

[0142] After calculating and generating the orientation feature vector and displacement feature vector of the target object, the angle between the orientation feature vector and the displacement feature vector can be further calculated to determine the target moving direction of the target object according to the size of the angle, thereby ensuring the accuracy of determining the target moving direction of the target object.

[0143] In a specific implementation provided in this specification, determining the target moving direction of the target object relative to itself according to the orientation feature vector and the displacement feature vector includes:

[0144] Calculating a relative travel angle between the orientation characteristic vector and the displacement characteristic vector;

[0145] Determining the preset angle interval to which the relative travel angle belongs in the preset angle interval set;

[0146] The target moving direction of the target object relative to itself is determined according to the relative moving angle and the preset angle interval.

[0147] Among them, the relative travel angle can be understood as the vector angle between the orientation feature vector and the displacement feature vector. The preset angle interval set refers to a set for storing multiple preset angle intervals, specifically including a first preset angle interval, a second preset angle interval and a third preset angle interval. The first preset angle interval, the second preset angle interval and the third preset angle interval are angle intervals of three different interval ranges, and the range sizes of the three preset angle intervals are: the first preset angle interval < the second preset angle interval < the third preset angle interval. For example, the first preset angle interval can be [0°, 85°], the second preset angle interval can be (85°, 95°], and the third preset angle interval can be (95°, 180°]. The preset angle interval is any angle interval stored in the preset angle interval set. It should be noted that the settings of the above three preset angle intervals are only used as examples and are not limited to the above examples.

[0148] Specifically, the relative travel angle between the orientation feature vector and the displacement feature vector is calculated according to the orientation feature vector and the displacement feature vector, and the arc cosine function can be used for calculation, specifically referring to the following formula 2:

[0149] Formula 2

[0150] in, is the relative travel angle between the orientation feature vector and the displacement feature vector. After the relative travel angle is calculated according to the above formula 2, the preset angle interval to which the relative travel angle belongs is determined in the preset angle interval set, so that the target travel direction of the target object can be determined according to the relative travel angle and the preset angle interval to which the relative travel angle belongs.

[0151] The following further explains the specific implementation method of determining the target travel direction of the target object based on the relative travel angle and the preset angle range.

[0152] In a specific implementation manner provided in this specification, the preset angle interval includes a first preset angle interval, a second preset angle interval or a third preset angle interval;

[0153] Determining the target moving direction of the target object relative to itself according to the relative moving angle and the preset angle interval includes:

[0154] When the relative travel angle is within a first preset angle interval, determining the target travel direction as a forward travel direction;

[0155] When the relative travel angle is within a second preset angle interval, determining the target travel direction to be a lateral travel direction;

[0156] When the relative travel angle is within a third preset angle interval, the target travel angle is determined to be a backward travel direction.

[0157] After determining the relative travel angle between the orientation feature vectors used for the displacement feature vectors, and the preset angle interval to which the relative travel angle belongs, the travel direction of the target object can be determined based on the relative travel angle and the preset angle interval to which it belongs. Since the preset angle interval set specifically includes the first preset angle interval, the second preset angle interval, and the third preset angle interval, the preset angle interval to which the relative travel angle belongs may be the first preset angle interval, the second preset angle interval, or the third preset angle interval, based on which, the three preset angle intervals can be analyzed and judged respectively.

[0158] Specifically, if the relative travel angle between the orientation characteristic vector and the displacement characteristic vector is in the first preset angle interval, it can be determined that the target travel direction of the target object is the forward travel direction, that is, the target object is moving forward relative to itself; if the relative travel angle between the orientation characteristic vector and the displacement characteristic vector is in the second preset angle interval, it can be determined that the target travel direction of the target object is the lateral travel direction, that is, the target object is moving laterally relative to itself; if the relative travel angle between the orientation characteristic vector and the displacement characteristic vector is in the third preset angle interval, it can be determined that the target travel direction of the target object is the backward travel direction, that is, the target object is moving backward relative to itself.

[0159] Using the above example, if the relative travel angle Located in the first preset angle interval [0°, 85°], that is , then the target moving direction of the target object is determined to be the forward moving direction; if the relative moving angle Located in the second preset angle interval (85°, 95°], that is , then the target moving direction of the target object is determined to be the lateral moving direction; if the relative moving angle Located in the third preset angle interval (95°, 180°], that is , then it is determined that the target moving direction of the target object is the backward moving direction.

[0160] In one embodiment provided in the present specification, by calculating the relative travel angle between the orientation characteristic vector and the displacement characteristic vector of the target object, and after determining the preset angle range of the relative travel angle, the target travel direction of the target object is determined according to the relative travel angle and the preset angle range corresponding to the relative travel angle, thereby improving the efficiency and accuracy of determining the target travel direction of the target object.

[0161] In actual applications, when pedestrians move in the forward direction, they may move in the forward direction or in the diagonal forward direction. Therefore, after determining that the target moving direction of the target object is the forward moving direction, the forward moving direction may be further subdivided to accurately determine the target moving direction of the target object.

[0162] In a specific implementation provided in this specification, when the relative travel angle is within the first preset angle interval, after determining that the target travel direction is a forward travel direction, the method further includes:

[0163] Determining a first forward angle interval and a first oblique angle interval in the first preset angle interval;

[0164] When the relative travel angle is within the first positive angle interval, determining the forward travel direction to be a positive forward direction;

[0165] When the relative travel angle is within the first oblique angle interval, the forward travel direction is determined to be an oblique forward direction.

[0166] Among them, the first preset angle interval includes a first forward angle interval and a first oblique angle interval; the first forward angle interval is used to determine whether the target object is moving forward relative to itself; the first oblique angle interval is used to determine whether the target object is moving obliquely relative to itself.

[0167] Specifically, after determining that the target moving direction of the target object is the forward moving direction, a first positive angle interval and a first oblique angle interval can be determined in the first preset angle interval to determine whether the target object is moving forward relative to itself or moving obliquely relative to itself. If the relative moving angle is in the first positive angle interval, the forward moving direction of the target object is determined to be the forward moving direction, that is, the target object is moving forward relative to itself; if the relative moving angle is in the first oblique angle interval, the forward moving direction of the target object is determined to be the oblique moving direction, that is, the target object is moving obliquely relative to itself.

[0168] The above example is still used, and the first forward angle interval is determined to be [0°, 5°] in the first preset angle interval [0°, 85°], and the first oblique angle interval is determined to be (5°, 85°] as an example for explanation. If the relative travel angle Located in the first positive angle interval [0°, 5°], that is , determine that the target object's forward direction is specifically the forward direction; if the relative travel angle Located in the first oblique angle interval (5°, 85°], that is , determine that the forward moving direction of the target object is specifically an oblique moving direction.

[0169] In an embodiment provided in the present specification, the first preset angle interval is further subdivided into a first forward angle interval and a first oblique angle interval, and it is determined whether the relative travel angle specifically belongs to the first forward angle interval or the first oblique angle interval, and the forward travel direction of the target object relative to itself is further judged, thereby improving the accuracy of judging the travel direction of the target object.

[0170] Furthermore, if it is determined that the target object's forward moving direction is an oblique moving direction, since the oblique moving direction may specifically be moving obliquely to the right front relative to itself, or moving obliquely to the left front relative to itself, after determining that the target object's forward moving direction is an oblique moving direction, the target object's moving direction can be further determined.

[0171] Based on this, in a specific implementation provided in this specification, when the relative travel angle is within the first oblique angle interval, after determining that the forward travel direction is an oblique forward direction, the method further includes:

[0172] Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product;

[0173] When the direction of the target vector is consistent with the vertical axis direction of the coordinate system, determining the oblique advancing direction as a first oblique advancing direction;

[0174] When the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the oblique advancing direction is determined to be the second oblique advancing direction.

[0175] The target vector product specifically refers to the vector outer product between the orientation eigenvector and the displacement eigenvector, that is, the vector cross product. The target vector direction is the vector direction of the target vector product. The first oblique advancing direction refers to the advancing direction to the right obliquely forward relative to the target object; the second oblique advancing direction refers to the advancing direction to the left obliquely forward relative to the target object.

[0176] Specifically, after determining that the forward direction of the target object is an oblique forward direction, the forward direction of the target object can be further judged by calculating the target vector product between the orientation feature vector and the displacement feature vector. Calculate the target vector product between the orientation feature vector and the displacement feature vector, and determine the target vector direction of the target vector product. After determining the vertical axis direction of the coordinate system according to the right-hand rule, determine whether the target vector direction of the target vector product is consistent with the vertical axis direction of the coordinate system. If they are consistent (that is, the target vector direction points to the positive direction of the vertical axis of the coordinate system), the oblique forward direction of the target object is determined to be the first oblique forward direction, that is, the target object moves obliquely to the right front relative to itself; if they are inconsistent (that is, the target vector direction points to the negative direction of the vertical axis of the coordinate system), the oblique forward direction of the target object is determined to be the second oblique forward direction, that is, the target object moves obliquely to the left front relative to itself.

[0177] In an embodiment provided in the present specification, by calculating the target vector product between the orientation feature vector and the displacement feature vector, and judging whether the target vector direction of the target vector product is consistent with the vertical axis direction of the coordinate system, the target object is determined to be moving in an oblique forward direction, specifically moving obliquely to the right front relative to itself or moving obliquely to the left front relative to itself, thereby achieving a subdivision of the oblique forward direction.

[0178] Corresponding to the above-mentioned subdivision of the forward traveling direction, in actual applications, when pedestrians are moving in the backward traveling direction, they may also be moving in the forward backward direction or in the oblique backward direction. Therefore, after determining that the target traveling direction of the target object is the backward traveling direction, the backward traveling direction can be further subdivided to accurately determine the target traveling direction of the target object.

[0179] In a specific implementation provided in this specification, when the relative travel angle is within the third preset angle interval, after determining that the target travel angle is a backward travel direction, the method further includes:

[0180] Determining a second forward angle interval and a second oblique angle interval in the third preset angle interval;

[0181] When the relative travel angle is within the second positive angle interval, determining the backward travel direction to be a positive backward direction;

[0182] When the relative travel angle is within the second oblique angle interval, the backward travel direction is determined to be an oblique backward direction.

[0183] Among them, the third preset angle interval includes a second forward angle interval and a second oblique angle interval; the second forward angle interval is used to determine whether the target object is moving forward backward relative to itself; the second oblique angle interval is used to determine whether the target object is moving obliquely backward relative to itself.

[0184] Specifically, after determining that the target moving direction of the target object is the backward moving direction, a second positive angle interval and a second oblique angle interval can be determined in the third preset angle interval to determine whether the target object is moving forward backward relative to itself or moving obliquely backward relative to itself. If the relative moving angle is in the second positive angle interval, the backward moving direction of the target object is determined to be the positive backward direction, that is, the target object is moving forward backward relative to itself; if the relative moving angle is in the second oblique angle interval, the backward moving direction of the target object is determined to be the oblique backward direction, that is, the target object is moving obliquely backward relative to itself.

[0185] Continuing with the above example, the second forward angle interval is determined to be (175°, 180°] in the third preset angle interval (95°, 180°], and the second oblique angle interval is determined to be (95°, 175°] as an example for explanation. If the relative travel angle Located in the second positive angle interval (175°, 180°], that is , determine that the target object's backward direction is specifically the forward backward direction; if the relative travel angle Located in the second oblique angle interval (95°, 175°], that is , determine that the backward direction of the target object is specifically an oblique backward direction.

[0186] In an embodiment provided in the present specification, the third preset angle interval is subdivided into a second forward angle interval and a second oblique angle interval, and it is determined that the relative travel angle specifically belongs to the second forward angle interval or the second oblique angle interval, and the backward travel direction of the target object relative to itself is further judged, thereby improving the accuracy of judging the travel direction of the target object.

[0187] Furthermore, if it is determined that the target object's backward movement direction is an oblique backward movement direction, since the oblique backward movement direction may specifically be moving obliquely to the right rearward relative to itself, or moving obliquely to the left rearward relative to itself, after determining that the target object's backward movement direction is an oblique backward movement direction, the target object's movement direction can be further determined.

[0188] Based on this, in a specific implementation provided in this specification, when the relative travel angle is within the second oblique angle interval, after determining that the backward travel direction is the oblique backward direction, the method further includes:

[0189] Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product;

[0190] In a case where the target vector direction is consistent with the vertical axis direction of the coordinate system, determining the oblique backward direction as a first oblique backward direction;

[0191] When the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the oblique backward direction is determined to be the second oblique backward direction.

[0192] The first oblique backward direction refers to the backward direction obliquely to the right rearward relative to the target object; the second oblique backward direction refers to the backward direction obliquely to the left rearward relative to the target object.

[0193] Specifically, similar to the above-mentioned subdivision of the oblique forward direction, after determining that the target object's backward direction is the oblique backward direction, the target object's direction of travel can also be further judged by calculating the target vector product between the orientation feature vector and the displacement feature vector. Calculate the target vector product between the orientation feature vector and the displacement feature vector, and determine the target vector direction of the target vector product. After determining the vertical axis direction of the coordinate system according to the right-hand rule, determine whether the target vector direction of the target vector product is consistent with the vertical axis direction of the coordinate system. If they are consistent (that is, the target vector direction points to the positive direction of the vertical axis of the coordinate system), then the oblique backward direction of the target object is determined to be the first oblique backward direction, that is, the target object retreats obliquely to the right relative to itself; if they are inconsistent (that is, the target vector direction points to the negative direction of the vertical axis of the coordinate system), then the oblique backward direction of the target object is determined to be the second oblique backward direction, that is, the target object retreats obliquely to the left relative to itself.

[0194] In one embodiment provided in the present specification, by calculating the target vector product between the orientation feature vector and the displacement feature vector, and judging whether the target vector direction of the target vector product is consistent with the vertical axis direction of the coordinate system, the target object is able to subdivide the oblique backward direction by determining whether it is moving obliquely to the right or to the left relative to itself during the process of moving in the oblique backward direction.

[0195] Furthermore, in actual applications, while pedestrians are moving in the lateral direction, they may also be moving in the right lateral direction or the left lateral direction. Therefore, after determining that the target travel direction of the target object is the lateral travel direction, the lateral travel direction can be further subdivided to accurately determine the target travel direction of the target object.

[0196] In a specific implementation provided in this specification, when the relative travel angle is within the second preset angle interval, after determining that the target travel direction is a lateral travel direction, the method further includes:

[0197] Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product;

[0198] In a case where the target vector direction is consistent with the vertical axis direction of the coordinate system, determining the lateral travel direction as a first lateral travel direction;

[0199] In a case where the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the lateral travel direction is determined to be a second lateral travel direction.

[0200] The first lateral moving direction refers to a direction moving toward the right relative to the target object; the second lateral moving direction refers to a direction moving toward the left relative to the target object.

[0201] Specifically, similar to the above method, after determining that the target travel direction of the target object is the lateral travel direction, the lateral travel direction of the target object is further judged by calculating the target vector product between the orientation feature vector and the displacement feature vector. Calculate the target vector product between the orientation feature vector and the displacement feature vector, and determine the target vector direction of the target vector product. After determining the vertical axis direction of the coordinate system according to the right-hand rule, determine whether the target vector direction of the target vector product is consistent with the vertical axis direction of the coordinate system. If they are consistent (that is, the target vector direction points to the positive direction of the vertical axis of the coordinate system), the lateral travel direction of the target object is determined to be the first lateral travel direction, that is, the target object moves laterally to the right relative to itself; if they are inconsistent (that is, the target vector direction points to the negative direction of the vertical axis of the coordinate system), the lateral travel direction of the target object is determined to be the second lateral travel direction, that is, the target object moves laterally to the left relative to itself.

[0202] In a specific embodiment provided in the present specification, by calculating the target vector product between the orientation feature vector and the displacement feature vector, and judging whether the target vector direction of the target vector product is consistent with the vertical axis direction of the coordinate system, the target object is able to subdivide the lateral travel direction by determining whether it is moving laterally to the right relative to itself or moving laterally to the left relative to itself during the process of moving in the lateral travel direction.

[0203] The method for determining the direction of an object moving relative to itself provided in the present specification includes: acquiring an image to be identified; identifying a target object in the image to be identified, and obtaining target detection frame information and target detection feature point information of the target object; generating target tracking feature point information of the target object in the image to be identified based on the target detection frame information and the target detection feature point information; generating an orientation feature vector and a displacement feature vector of the target object based on the target tracking feature point information; and determining the target direction of the target object moving relative to itself based on the orientation feature vector and the displacement feature vector.

[0204] One or more embodiments of the present specification realize that target detection is performed on the image to be identified to obtain the target detection frame information and target detection feature point information of the target object in the image to be identified, and the target tracking frame information is generated based on the target detection frame information and its corresponding reference detection frame information, thereby determining the target tracking feature point information of the target object. Based on the reference tracking feature point information and the target tracking feature point information, the orientation feature vector and the displacement feature vector of the target object are calculated, and the target travel direction of the target object is determined by calculating the relative travel angle between the orientation feature vector and the displacement feature vector, thereby improving the accuracy of determining the target travel direction of the target object. In addition, the body rotation angle of the target object can also be determined based on the relative travel angle between the orientation feature vector and the displacement feature vector, thereby improving the accuracy of determining the body rotation angle.

[0205] The following combination Figure 4 Taking the application of the method for determining the direction of an object relative to itself provided in this specification in a subway gate scene as an example, the method for determining the direction of an object relative to itself is further described. Figure 4 FIG. 1 shows a flow chart of a processing process of a method for determining a moving direction of an object relative to itself provided by an embodiment of the present specification. Figure 4As shown, a color camera is used to obtain an image to be identified from the top perspective corresponding to the subway gate, and the image to be identified is input into the target detection model to obtain the target detection frame information and target detection feature point information output by the target detection model, wherein the target detection frame information includes the target detection frame identifier and the target detection frame position information. The reference detection frame position information is obtained from the obtained reference detection frame information, and the reference detection frame position information and the target detection frame position information are calculated using a tracking algorithm to obtain the target tracking frame position information of the pedestrian. Furthermore, based on the correspondence between the target tracking frame and the target detection frame, and the correspondence between the target detection feature points used for the target detection frame, the target tracking feature points are matched to the target tracking frame, and the target tracking feature point information of the target tracking feature points is determined. The tracking feature points specifically include the pedestrian's human body center point, back spine point, chest center point, left shoulder point and right shoulder point feature points. The target tracking feature points are the tracking feature points of the pedestrian in the image to be identified, and the reference tracking feature points are the tracking feature points of the pedestrian in the reference identification image.

[0206] Furthermore, after obtaining the target tracking feature point information, the pedestrian's orientation feature vector is generated according to the target tracking feature point information, specifically, the longitudinal feature vector is calculated and generated according to the position information of the pedestrian's back spine point and the position information of the chest center point, and the transverse feature vector is calculated and generated according to the position information of the pedestrian's left shoulder point and the right shoulder point. The longitudinal feature vector and the transverse feature vector are unitized to obtain the longitudinal unit vector (that is, the second orientation feature vector) and the transverse unit vector respectively, and the transverse unit vector is rotated using a preset rotation matrix to obtain the first orientation feature vector. The pedestrian's orientation feature vector is generated according to the first orientation feature vector and the second orientation feature vector.

[0207] A displacement feature vector of a pedestrian is generated according to the reference tracking feature point information and the target tracking feature point information. Specifically, a candidate displacement feature vector is calculated and generated according to the position information of the center point of the human body in the reference recognition image and the position information of the center point of the human body in the image to be recognized. The candidate displacement feature vector is normalized to obtain a displacement feature vector, that is, a unit vector of the candidate displacement feature vector.

[0208] Furthermore, an arccosine function is used to calculate a relative travel angle between the orientation feature vector and the displacement feature vector, and the travel direction of the pedestrian is determined according to the relative travel angle. Specifically, if the relative travel angle is between [0°, 5°], the pedestrian's target travel direction is determined to be forward relative to itself; if the relative travel angle is between (5°, 85°], the target vector product between the orientation eigenvector and the displacement eigenvector is calculated, and the target vector direction of the target vector product is determined. After the direction of the vertical axis of the coordinate system is determined according to the right-hand rule, if the target vector direction points to the positive direction of the vertical axis of the coordinate system, it means that the pedestrian's target travel direction is forward to the right relative to itself. If the target vector direction points to the negative direction of the vertical axis of the coordinate system, it means that the pedestrian's target travel direction is forward to the left relative to itself. If the relative travel angle is between (85°, 95°], the pedestrian's target travel direction is determined to be sideways relative to itself. Further, the target vector product between the orientation eigenvector and the displacement eigenvector can be calculated, and the target vector direction of the target vector product can be determined. After the direction of the vertical axis of the coordinate system is determined according to the right-hand rule, if the target vector direction points to the positive direction of the vertical axis of the coordinate system, direction, it means that the pedestrian's target travel direction is to the right relative to itself. If the target vector direction points to the negative direction of the vertical axis of the coordinate system, it means that the pedestrian's target travel direction is to the left relative to itself. If the relative travel angle is between (95°, 175°], the target vector product between the orientation eigenvector and the displacement eigenvector is calculated in the same way as above, and the target vector direction of the target vector product is determined. After the direction of the vertical axis of the coordinate system is determined according to the right-hand rule, if the target vector direction points to the positive direction of the vertical axis of the coordinate system, it means that the pedestrian's target travel direction is to retreat diagonally to the right relative to itself. If the target vector direction points to the negative direction of the vertical axis of the coordinate system, it means that the pedestrian's target travel direction is to retreat diagonally to the left relative to itself. If the target travel angle is between (175°, 180°], it is determined that the pedestrian's target travel direction is to retreat positively relative to itself. The target travel direction of pedestrians in subway gates can be accurately determined by the above method for determining the direction of travel of an object relative to itself.

[0209] An embodiment of the present specification realizes that target detection is performed on an image to be identified to obtain target detection frame information and target detection feature point information of a pedestrian in the image to be identified, and target tracking frame information is generated based on the target detection frame information and its corresponding reference detection frame information, thereby determining the target tracking feature point information of the pedestrian. Based on the reference tracking feature point information and the target tracking feature point information, the orientation feature vector and the displacement feature vector of the pedestrian are calculated, and the relative travel angle between the orientation feature vector and the displacement feature vector is calculated to achieve the determination of the target travel direction of the pedestrian relative to itself, thereby improving the accuracy of determining the target travel direction of the pedestrian relative to itself.

[0210] Corresponding to the above method embodiment, this specification also provides an embodiment of a device for determining a direction of travel of an object relative to itself, Figure 5 FIG. 2 shows a schematic diagram of a device for determining the direction of an object relative to itself provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0211] An acquisition module 502 is configured to acquire an image to be recognized;

[0212] The recognition module 504 is configured to recognize a target object in the image to be recognized, and obtain target detection frame information and target detection feature point information of the target object;

[0213] A first generating module 506 is configured to generate target tracking feature point information of the target object in the to-be-recognized image according to the target detection frame information and the target detection feature point information;

[0214] A second generating module 508 is configured to generate a direction feature vector and a displacement feature vector of the target object according to the target tracking feature point information;

[0215] The determination module 510 is configured to determine the target moving direction of the target object relative to itself according to the orientation feature vector and the displacement feature vector.

[0216] Optionally, the second generating module 508 is further configured to:

[0217] Acquire reference tracking feature point information of the target object in the reference recognition image;

[0218] Generating a direction feature vector of the target object according to the target tracking feature point information;

[0219] A displacement feature vector of the target object is generated according to the target tracking feature point information and the reference tracking feature point information.

[0220] Optionally, the second generating module 508 is further configured to:

[0221] Acquire target horizontal feature point information and target vertical feature point information from the target tracking feature point information;

[0222] Generating a first orientation feature vector of the target object according to the target lateral feature point information;

[0223] Generating a second orientation feature vector of the target object according to the target longitudinal feature point information;

[0224] An orientation feature vector of the target object is generated according to the first orientation feature vector and the second orientation feature vector.

[0225] Optionally, the second generating module 508 is further configured to:

[0226] Determining left shoulder point feature point information and right shoulder point feature point information in the target lateral feature point information;

[0227] Calculating a lateral feature vector according to the feature point information of the left shoulder point and the feature point information of the right shoulder point;

[0228] Unitizing the transverse eigenvector to obtain a transverse unit vector of the transverse eigenvector;

[0229] The lateral unit vector is rotated according to a preset rotation matrix to generate a first orientation feature vector of the target object.

[0230] Optionally, the second generating module 508 is further configured to:

[0231] Determining chest feature point information and spine feature point information in the target longitudinal feature point information;

[0232] Calculating a longitudinal feature vector according to the chest feature point information and the spine feature point information;

[0233] The longitudinal feature vector is normalized to generate a second orientation feature vector of the target object.

[0234] Optionally, the second generating module 508 is further configured to:

[0235] Acquire target center feature point information from the target tracking feature point information, and acquire reference center feature point information from the reference tracking feature point information;

[0236] Calculating a candidate displacement feature vector according to the target center feature point information and the reference center feature point information;

[0237] The candidate displacement feature vectors are normalized to generate a displacement feature vector of the target object.

[0238] Optionally, the determining module 510 is further configured to:

[0239] Calculating a relative travel angle between the orientation characteristic vector and the displacement characteristic vector;

[0240] Determining the preset angle interval to which the relative travel angle belongs in the preset angle interval set;

[0241] The target moving direction of the target object relative to itself is determined according to the relative moving angle and the preset angle interval.

[0242] Optionally, the preset angle interval includes a first preset angle interval, a second preset angle interval or a third preset angle interval;

[0243] The determination module 510 is further configured to:

[0244] When the relative travel angle is within a first preset angle interval, determining the target travel direction as a forward travel direction;

[0245] When the relative travel angle is within a second preset angle interval, determining the target travel direction to be a lateral travel direction;

[0246] When the relative travel angle is within a third preset angle interval, the target travel angle is determined to be a backward travel direction.

[0247] Optionally, the device further includes a first direction determining module, configured to:

[0248] Determining a first forward angle interval and a first oblique angle interval in the first preset angle interval;

[0249] When the relative travel angle is within the first positive angle interval, determining the forward travel direction to be a positive forward direction;

[0250] When the relative travel angle is within the first oblique angle interval, the forward travel direction is determined to be an oblique forward direction.

[0251] Optionally, the device further includes a first sub-direction determining module, configured to:

[0252] Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product;

[0253] When the direction of the target vector is consistent with the vertical axis direction of the coordinate system, determining the oblique advancing direction as a first oblique advancing direction;

[0254] When the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the oblique advancing direction is determined to be the second oblique advancing direction.

[0255] Optionally, the device further includes a second direction determining module, configured to:

[0256] Determining a second forward angle interval and a second oblique angle interval in the third preset angle interval;

[0257] When the relative travel angle is within the second positive angle interval, determining the backward travel direction to be a positive backward direction;

[0258] When the relative travel angle is within the second oblique angle interval, the backward travel direction is determined to be an oblique backward direction.

[0259] Optionally, the device further includes a second sub-direction determining module, configured to:

[0260] Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product;

[0261] In a case where the target vector direction is consistent with the vertical axis direction of the coordinate system, determining the oblique backward direction as a first oblique backward direction;

[0262] When the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the oblique backward direction is determined to be the second oblique backward direction.

[0263] Optionally, the device further includes a third direction determination module, configured to:

[0264] Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product;

[0265] In a case where the target vector direction is consistent with the vertical axis direction of the coordinate system, determining the lateral travel direction as a first lateral travel direction;

[0266] In a case where the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the lateral travel direction is determined to be a second lateral travel direction.

[0267] Optionally, the identification module 504 is further configured to:

[0268] Inputting the image to be recognized into a target detection model;

[0269] Obtain target detection frame information and target detection feature point information output by the target detection model.

[0270] Optionally, the device further comprises a training module configured to:

[0271] Acquire sample training data, wherein the sample training data includes an image to be trained, sample detection frame information of the image to be trained, and sample detection feature point information;

[0272] Inputting the image to be trained into the target detection model to obtain predicted detection frame information and predicted detection feature point information of the image to be trained;

[0273] Calculate a first loss value according to the predicted detection frame information and the sample detection frame information, and calculate a second loss value according to the predicted detection feature point information and the sample detection feature point information;

[0274] According to the first loss value and the second loss value, the model parameters of the target detection model are adjusted, and the target detection model is continuously trained until a training stop condition is reached.

[0275] Optionally, the first generating module 506 is further configured to:

[0276] Obtaining reference detection frame information of the target object in the reference recognition image;

[0277] Generating target tracking frame information of the target object according to the reference detection frame information and the target detection frame information;

[0278] The target tracking feature point information of the target object in the image to be identified is obtained according to the target detection frame information, the target tracking frame information and the target detection feature point information.

[0279] Optionally, the first generating module 506 is further configured to:

[0280] Acquire reference detection frame position information from the reference detection frame information, and acquire target detection frame position information and a target detection frame identifier from the target detection frame information;

[0281] Calculating target tracking frame position information of the target object according to the reference detection frame position information, the target detection frame position information and the tracking algorithm;

[0282] Determine the target detection frame identifier as a target tracking frame identifier;

[0283] Based on the target tracking frame identifier and the target tracking frame position information, target tracking frame information of the target object is generated.

[0284] Optionally, the first generating module 506 is further configured to:

[0285] Determining a first corresponding relationship between the target detection frame and the target tracking frame according to the target detection frame information and the target tracking frame information;

[0286] Determining a second corresponding relationship between the target detection frame and the target detection feature point according to the target detection frame information and the target detection feature point information;

[0287] According to the first corresponding relationship and the second corresponding relationship, the target tracking feature points corresponding to the target tracking frame are matched to obtain the target tracking feature point information of the target object in the image to be identified.

[0288] The device for determining the direction of an object moving relative to itself provided in the present specification includes: an acquisition module, configured to acquire an image to be identified; an identification module, configured to identify a target object in the image to be identified, and obtain target detection frame information and target detection feature point information of the target object; a first generation module, configured to generate target tracking feature point information of the target object in the image to be identified based on the target detection frame information and the target detection feature point information; a second generation module, configured to generate an orientation feature vector and a displacement feature vector of the target object based on the target tracking feature point information; and a determination module, configured to determine the target moving direction of the target object relative to itself based on the orientation feature vector and the displacement feature vector.

[0289] One or more embodiments of the present specification realize that target detection is performed on the image to be identified to obtain the target detection frame information and target detection feature point information of the target object in the image to be identified, and the target tracking frame information is generated based on the target detection frame information and its corresponding reference detection frame information, thereby determining the target tracking feature point information of the target object. Based on the reference tracking feature point information and the target tracking feature point information, the orientation feature vector and the displacement feature vector of the target object are calculated, and the target travel direction of the target object is determined by calculating the relative travel angle between the orientation feature vector and the displacement feature vector, thereby improving the accuracy of determining the target travel direction of the target object. In addition, the body rotation angle of the target object can also be determined based on the relative travel angle between the orientation feature vector and the displacement feature vector, thereby improving the accuracy of determining the body rotation angle.

[0290] The above is a schematic scheme of a device for determining the direction of an object relative to itself in this embodiment. It should be noted that the technical scheme of the device for determining the direction of an object relative to itself and the technical scheme of the method for determining the direction of an object relative to itself are of the same concept, and the details not described in detail in the technical scheme of the device for determining the direction of an object relative to itself can be referred to the description of the technical scheme of the method for determining the direction of an object relative to itself.

[0291] Figure 6 The structure block diagram of a computing device 600 provided according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0292] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0293] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0294] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 600 may also be a mobile or stationary server.

[0295] When the processor 620 executes the computer program / instructions, the steps of the method for determining the direction of movement of an object relative to itself are implemented.

[0296] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the method for determining the direction of an object relative to itself are of the same concept, and the details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the method for determining the direction of an object relative to itself.

[0297] An embodiment of the present specification also provides a computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps of the method for determining the direction of movement of an object relative to itself as described above.

[0298] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the method for determining the direction of travel of an object relative to itself are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the method for determining the direction of travel of an object relative to itself.

[0299] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for determining the direction of travel of an object relative to itself.

[0300] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the method for determining the direction of an object relative to itself are of the same concept, and the details not described in detail in the technical scheme of the computer program product can be found in the description of the technical scheme of the method for determining the direction of an object relative to itself.

[0301] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0302] The computer program / instruction includes a computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0303] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0304] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0305] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for determining the direction of an object relative to itself, characterized in that: include: Acquire an image to be identified, wherein the image to be identified is acquired based on an image acquisition device, and the image acquisition device adopts a top-down perspective; Identify a target object in the image to be identified, and obtain target detection frame information and target detection feature point information of the target object; Generating target tracking feature point information of the target object in the image to be identified according to the target detection frame information and the target detection feature point information; Generating an orientation feature vector and a displacement feature vector of the target object according to the target tracking feature point information; Calculate the relative travel angle between the orientation feature vector and the displacement feature vector; determine the preset angle interval to which the relative travel angle belongs in a preset angle interval set; and determine the target travel direction of the target object relative to itself based on the relative travel angle and the preset angle interval.

2. The method according to claim 1, characterized in that Generating the orientation feature vector and displacement feature vector of the target object according to the target tracking feature point information includes: Acquire reference tracking feature point information of the target object in the reference recognition image; Generating a direction feature vector of the target object according to the target tracking feature point information; A displacement feature vector of the target object is generated according to the target tracking feature point information and the reference tracking feature point information.

3. The method according to claim 2, characterized in that Generating a direction feature vector of the target object according to the target tracking feature point information includes: Acquire target horizontal feature point information and target vertical feature point information from the target tracking feature point information; Generating a first orientation feature vector of the target object according to the target lateral feature point information; Generating a second orientation feature vector of the target object according to the target longitudinal feature point information; An orientation feature vector of the target object is generated according to the first orientation feature vector and the second orientation feature vector.

4. The method according to claim 3, characterized in that Generating a first orientation feature vector of the target object according to the target lateral feature point information includes: Determining left shoulder point feature point information and right shoulder point feature point information in the target lateral feature point information; Calculating a lateral feature vector according to the feature point information of the left shoulder point and the feature point information of the right shoulder point; Unitizing the transverse eigenvector to obtain a transverse unit vector of the transverse eigenvector; The lateral unit vector is rotated according to a preset rotation matrix to generate a first orientation feature vector of the target object.

5. The method according to claim 3, characterized in that Generating a second orientation feature vector of the target object according to the target longitudinal feature point information includes: Determining chest feature point information and spine feature point information in the target longitudinal feature point information; Calculating a longitudinal feature vector according to the chest feature point information and the spine feature point information; The longitudinal feature vector is normalized to generate a second orientation feature vector of the target object.

6. The method according to claim 2, characterized in that Generating a displacement feature vector of the target object according to the target tracking feature point information and the reference tracking feature point information, including: Acquire target center feature point information from the target tracking feature point information, and acquire reference center feature point information from the reference tracking feature point information; Calculating a candidate displacement feature vector according to the target center feature point information and the reference center feature point information; The candidate displacement feature vectors are normalized to generate a displacement feature vector of the target object.

7. The method according to claim 1, characterized in that The preset angle interval includes a first preset angle interval, a second preset angle interval or a third preset angle interval; Determining the target moving direction of the target object relative to itself according to the relative moving angle and the preset angle interval includes: When the relative travel angle is within a first preset angle interval, determining the target travel direction as a forward travel direction; When the relative travel angle is within a second preset angle interval, determining the target travel direction to be a lateral travel direction; When the relative travel angle is within a third preset angle interval, the target travel angle is determined to be a backward travel direction.

8. The method according to claim 7, characterized in that When the relative travel angle is within the first preset angle interval, after determining that the target travel direction is a forward travel direction, the method further includes: Determining a first forward angle interval and a first oblique angle interval in the first preset angle interval; When the relative travel angle is within the first positive angle interval, determining the forward travel direction to be a positive forward direction; When the relative travel angle is within the first oblique angle interval, the forward travel direction is determined to be an oblique forward direction.

9. The method according to claim 8, characterized in that When the relative travel angle is within the first oblique angle interval, after determining that the forward travel direction is an oblique forward direction, the method further includes: Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product; When the direction of the target vector is consistent with the vertical axis direction of the coordinate system, determining the oblique advancing direction as a first oblique advancing direction; When the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the oblique advancing direction is determined to be the second oblique advancing direction.

10. The method according to claim 7, characterized in that When the relative travel angle is within the third preset angle interval, after determining that the target travel angle is a backward travel direction, the method further includes: Determining a second forward angle interval and a second oblique angle interval in the third preset angle interval; When the relative travel angle is within the second positive angle interval, determining the backward travel direction to be a positive backward direction; When the relative travel angle is within the second oblique angle interval, the backward travel direction is determined to be an oblique backward direction.

11. The method according to claim 10, characterized in that When the relative travel angle is within the second oblique angle interval, after determining that the backward travel direction is an oblique backward direction, the method further includes: Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product; In a case where the target vector direction is consistent with the vertical axis direction of the coordinate system, determining the oblique backward direction as a first oblique backward direction; When the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the oblique backward direction is determined to be the second oblique backward direction.

12. The method according to claim 7, characterized in that When the relative travel angle is within the second preset angle interval, after determining that the target travel direction is a lateral travel direction, the method further includes: Calculating a target vector product between the orientation feature vector and the displacement feature vector, and determining a target vector direction of the target vector product; In a case where the target vector direction is consistent with the vertical axis direction of the coordinate system, determining the lateral travel direction as a first lateral travel direction; In a case where the target vector direction is inconsistent with the vertical axis direction of the coordinate system, the lateral travel direction is determined to be a second lateral travel direction.

13. The method according to claim 1, characterized in that Identifying a target object in the image to be identified and obtaining target detection frame information and target detection feature point information of the target object include: Inputting the image to be recognized into a target detection model; Obtain target detection frame information and target detection feature point information output by the target detection model.

14. The method according to claim 13, characterized in that The target detection model is trained based on the following method: Acquire sample training data, wherein the sample training data includes an image to be trained, sample detection frame information of the image to be trained, and sample detection feature point information; Inputting the image to be trained into the target detection model to obtain predicted detection frame information and predicted detection feature point information of the image to be trained; Calculate a first loss value according to the predicted detection frame information and the sample detection frame information, and calculate a second loss value according to the predicted detection feature point information and the sample detection feature point information; According to the first loss value and the second loss value, the model parameters of the target detection model are adjusted, and the target detection model is continuously trained until a training stop condition is reached.

15. The method according to claim 1, wherein: Generating target tracking feature point information of the target object in the image to be identified according to the target detection frame information and the target detection feature point information, including: Obtaining reference detection frame information of the target object in the reference recognition image; Generating target tracking frame information of the target object according to the reference detection frame information and the target detection frame information; The target tracking feature point information of the target object in the image to be identified is obtained according to the target detection frame information, the target tracking frame information and the target detection feature point information.

16. The method according to claim 15, characterized in that Generating target tracking frame information of the target object according to the reference detection frame information and the target detection frame information, including: Acquire reference detection frame position information from the reference detection frame information, and acquire target detection frame position information and a target detection frame identifier from the target detection frame information; Calculating target tracking frame position information of the target object according to the reference detection frame position information, the target detection frame position information and the tracking algorithm; Determine the target detection frame identifier as a target tracking frame identifier; Based on the target tracking frame identifier and the target tracking frame position information, target tracking frame information of the target object is generated.

17. The method according to claim 15, characterized in that Obtaining target tracking feature point information of the target object in the image to be identified according to the target detection frame information, the target tracking frame information and the target detection feature point information, including: Determining a first corresponding relationship between the target detection frame and the target tracking frame according to the target detection frame information and the target tracking frame information; Determining a second corresponding relationship between the target detection frame and the target detection feature point according to the target detection frame information and the target detection feature point information; According to the first corresponding relationship and the second corresponding relationship, the target tracking feature points corresponding to the target tracking frame are matched to obtain the target tracking feature point information of the target object in the image to be identified.

18. A device for determining the direction of an object relative to itself, characterized in that: include: An acquisition module is configured to acquire an image to be recognized, wherein the image to be recognized is acquired based on an image acquisition device, and the image acquisition device adopts a top-down perspective; A recognition module is configured to recognize a target object in the image to be recognized, and obtain target detection frame information and target detection feature point information of the target object; A first generating module is configured to generate target tracking feature point information of the target object in the image to be identified according to the target detection frame information and the target detection feature point information; A second generating module is configured to generate a direction feature vector and a displacement feature vector of the target object according to the target tracking feature point information; A determination module is configured to calculate the relative travel angle between the orientation feature vector and the displacement feature vector; determine the preset angle interval to which the relative travel angle belongs in a preset angle interval set; and determine the target travel direction of the target object relative to itself based on the relative travel angle and the preset angle interval.

19. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. It is characterized in that when the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 17 are implemented.

20. A computer-readable storage medium storing a computer program / instruction, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

21. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

Images