Target following method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a target following method and device, electronic equipment and a medium. The target following method comprises the steps of collecting target smell information and an initial smell concentration value of a target object; performing target object detection according to the real-time image data; under the condition that the target object is detected, generating a first moving track of the movable equipment according to the position information of the target object, and controlling the movable equipment to follow and move according to the first moving track; and under the condition that the target object is not detected, according to the real-time odor concentration value of each grid area in the odor concentration grid map, determining position information of a plurality of target grid areas, generating a second moving track of the movable equipment, and controlling the movable equipment to follow and move according to the second moving track, the moving track of the movable equipment is generated according to the target object detection condition, so that the movable equipment is controlled to follow and move according to the moving track.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and particularly to a method, device, electronic device and medium for target following. Background Art

[0002] Target tracking technology, as an important technology in the fields of computer vision and machine learning, has been applied in many fields such as security monitoring and driverless in recent years.

[0003] However, existing target tracking technologies rely to a great extent on sensors such as vision or sound, which causes them to be easily affected by environmental factors such as light and noise in practical applications. For example, in an environment with poor lighting conditions or a large amount of noise, there may be cases of incorrect tracking or loss of the target, thus affecting the stability and reliability of the system.

[0004] In addition, existing target tracking technologies also pose certain risks in terms of privacy protection. During the process of implementing tracking, a large amount of personal information is often collected, such as the facial features and behavior habits of pedestrians. The leakage of this information not only violates personal privacy but may also pose security risks. Summary of the Invention

[0005] In view of the above problems, a method, device, electronic device and medium for target following are provided to overcome or at least partially solve the above problems, including:

[0006] A method for target following, which is applied to a movable device. The movable device is provided with a camera, a temperature sensor, a wind speed sensor and an odor sensor. The method includes:

[0007] In response to a data acquisition instruction, control the movable device to enter a target image acquisition mode. In the target image acquisition mode, collect target image features of a target object through the camera, and control the movable device to enter a target odor acquisition mode. In the target odor acquisition mode, collect target odor information and an initial odor concentration value of the target object through the odor sensor;

[0008] In response to a target following instruction, control the movable device to enter a target following mode. In the target following mode, collect real-time image data through the camera, and perform target object detection on the real-time image data based on the target image features;

[0009] When the target object is detected in the real-time image data, determine the position information of the target object. According to the position information of the target object, generate a first movement trajectory of the movable device, and control the movable device to perform following movement according to the first movement trajectory;

[0010] In the case where the target object is not detected in the real-time image data, the real-time temperature value of the temperature sensor, the real-time wind speed value collected by the wind speed sensor, and the real-time odor concentration values of multiple odor sampling points collected by the odor sensor;

[0011] According to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, and the initial odor concentration value, determine the distance values between the multiple odor sampling points and the mobile device, and establish an odor concentration grid map according to the distance values and the real-time odor concentration values; the odor concentration grid map includes multiple grid areas, each grid area has a corresponding odor sampling point, and the real-time odor concentration value of each grid area is the real-time odor concentration value of the corresponding odor sampling point;

[0012] According to the real-time odor concentration values of each grid area in the odor concentration grid map, determine multiple target grid areas from the multiple grid areas, generate a second movement trajectory of the mobile device according to the position information of the multiple target grid areas, and control the mobile device to perform following movement according to the second movement trajectory.

[0013] Optionally, before controlling the mobile device to perform following movement according to the second movement trajectory, it further includes:

[0014] Obtain the kinematic parameters of the mobile device, and optimize the second movement trajectory according to the kinematic parameters.

[0015] Optionally, the mobile device is provided with multiple odor sensors in different directions, and establishing an odor concentration grid map according to the distance values and the real-time odor concentration values includes:

[0016] For each odor sampling point, determine the target odor sensor that collects the real-time odor concentration value of the odor sampling point, and determine the position information of the odor sampling point according to the sampling direction information of the target odor sensor and the distance value;

[0017] Taking the mobile device as the center, divide the area within a preset range into multiple grid areas, and establish a corresponding relationship between the position information of the multiple odor sampling points and the position information of the multiple grid areas;

[0018] For each grid area, set the real-time odor concentration value of the corresponding odor sampling point as the real-time odor concentration value of the grid area to establish an odor concentration grid map.

[0019] Optionally, determining a plurality of target grid regions from the plurality of grid regions includes:

[0020] For each row of grid regions in the odor concentration grid map, determining the grid region with the maximum real-time odor concentration value as the target grid region in each row of grid regions.

[0021] Optionally, generating the second movement trajectory of the movable device according to the position information of the plurality of target grid regions includes:

[0022] Performing interpolation processing between the position information of two adjacent target grid regions to obtain a plurality of transition position information, and combining the position information of the plurality of target grid regions and the transition position information to generate the second movement trajectory of the movable device.

[0023] Optionally, determining the distance values between the plurality of odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, and the initial odor concentration value includes:

[0024] Obtaining a temperature influence coefficient, a wind speed influence coefficient, a distance attenuation exponent, an odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode;

[0025] Determining the distance values between the plurality of odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, the initial odor concentration value, the temperature influence coefficient, the wind speed influence coefficient, the distance attenuation exponent, the odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode.

[0026] Optionally, the movable device is further provided with a sound sensor. Before controlling the movable device to perform following movement according to the second movement trajectory, it further includes:

[0027] Obtaining audio data collected by the sound sensor for a plurality of audio sampling points;

[0028] Detecting the audio data, and when a target keyword is detected in the audio data, determining the position information of the target audio sampling point corresponding to the audio data;

[0029] Generating a third movement trajectory according to the position information of the target audio sampling point, and calculating the trajectory deviation between the second movement trajectory and the third movement trajectory;

[0030] When the trajectory deviation is less than a preset deviation value, performing the control to make the movable device perform following movement according to the second movement trajectory;

[0031] When the trajectory deviation is less than or equal to a preset deviation value, a warning message is fed back.

[0032] A device for target following, which is applied to a movable device. The movable device is provided with a camera, a temperature sensor, a wind speed sensor and an odor sensor. The device is used for:

[0033] In response to a data acquisition instruction, control the movable device to enter a target image acquisition mode. In the target image acquisition mode, collect target image features of a target object through the camera, and control the movable device to enter a target odor acquisition mode. In the target odor acquisition mode, collect target odor information and an initial odor concentration value of the target object through the odor sensor;

[0034] In response to a target following instruction, control the movable device to enter a target following mode. In the target following mode, collect real-time image data through the camera, and perform target object detection on the real-time image data by using the target image features;

[0035] When the target object is detected in the real-time image data, determine the position of the target object, generate a first movement trajectory of the movable device according to the position information of the target object, and control the movable device to perform following movement according to the first movement trajectory;

[0036] When the target object is not detected in the real-time image data, obtain the real-time temperature value through the temperature sensor, the real-time wind speed value through the wind speed sensor, and the real-time odor concentration values of a plurality of odor sampling points collected through the odor sensor;

[0037] According to the real-time temperature value, the real-time wind speed value, the real-time odor concentration values, and the initial odor concentration value, determine the distance values between the plurality of odor sampling points and the movable device, and establish an odor concentration grid map according to the distance values and the real-time odor concentration values; the odor concentration grid map includes a plurality of grid areas, each grid area has a corresponding odor sampling point, and the real-time odor concentration value of each grid area is the real-time odor concentration value of the corresponding odor sampling point;

[0038] According to the real-time odor concentration values of each grid area in the odor concentration grid map, determine a plurality of target grid areas from the plurality of grid areas, generate a second movement trajectory of the movable device according to the position information of the plurality of target grid areas, and control the movable device to perform following movement according to the second movement trajectory.

[0039] An electronic device includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor. When the computer program is executed by the processor, the above-described method is implemented.

[0040] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the above-described method is implemented.

[0041] The embodiments of the present invention have the following advantages:

[0042] In the embodiments of the present invention, by collecting the target odor information and the initial odor concentration value of the target object; and detecting the target object according to the real-time image data; when the target object is detected in the real-time image data, according to the position information of the target object, a first moving trajectory of the movable device is generated, and the movable device is controlled to follow the movement according to the first moving trajectory; when the target object is not detected in the real-time image data, according to the real-time odor concentration value of each grid area in the odor concentration grid map, the position information of multiple target grid areas is determined, a second moving trajectory of the movable device is generated, and the movable device is controlled to follow the movement according to the second moving trajectory, realizing generating the moving trajectory of the movable device according to the detection situation of the target object, so as to control the movable device to follow the movement according to the moving trajectory, ensuring the intelligent, continuous and accurate tracking of the movable device to the target object. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions of the present invention, the drawings required for the description of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 is a flowchart of the steps of a target following method provided by some embodiments of the present invention;

[0045] Figure 2 is a schematic diagram of an odor concentration grid map provided by some embodiments of the present invention;

[0046] Figure 3 is a schematic flowchart of another target object following provided by some embodiments of the present invention. Detailed Embodiments

[0047] To make the above objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0048] Referring to Figure 1 , a flowchart of steps of a target following method provided by some embodiments of the present invention is shown, which is applied to a movable device. The movable device is provided with a camera, a temperature sensor, a wind speed sensor, an odor sensor, and a sound sensor.

[0049] In practical applications, the movable device can be a device that can be carried and moved between different locations, such as a smart robot, a drone, etc. These devices can be equipped with a variety of sensors to collect and process different types of information about the surrounding environment.

[0050] Among them, the camera can be used to capture visual images of the surrounding environment; the temperature sensor can be used to measure and monitor the temperature of the surrounding environment; the wind speed sensor can be used to measure the speed and direction of air flow; the odor sensor (gas sensor) can be used to detect and identify specific gases or odors in the environment; the sound sensor can be used to capture and record sound signals in the environment.

[0051] Specifically, the following steps can be included:

[0052] Step 101, in response to a data collection instruction, control the movable device to enter a target image collection mode. In the target image collection mode, collect target image features of a target object through the camera, and control the movable device to enter a target odor collection mode. In the target odor collection mode, collect target odor information and an initial odor concentration value of the target object through the odor sensor.

[0053] As some examples, the movable device can receive an instruction to start or trigger a data collection task, and this instruction can come from a user, a remote control system, or any other triggering mechanism.

[0054] After the movable device receives a data collection instruction and enters the target image collection mode, the movable device can use the camera to capture an image of the target object. Among them, the target object can be a person, an animal, a vehicle, etc.; the image features of the target object can include the shape, color, texture, size, etc. of the object.

[0055] After the mobile device receives a data collection instruction and enters the target odor collection mode, the mobile device can use an odor sensor to detect and record the odor information and the initial odor concentration value emitted by the target object; among them, the odor information can include the type of gas, and the initial odor concentration value can be determined according to the odor intensity or quantity of the target object when it is first detected.

[0056] As some examples, before collecting the target odor information and the initial odor concentration value of the target object through the odor sensor, an odor source can be placed in the target object; among them, this odor source can be a mixture of one or more known gases, whose released odor characteristics are obvious and can be distinguished from the odor that the target object itself may generate; in this way, when the mobile device works in the target odor collection mode, it can more accurately identify the odor information of the target object and reduce the possibility of misjudgment; at the same time, using the known odor source can also calibrate the performance and accuracy of the odor sensor to ensure the reliability of data collection.

[0057] Step 102, in response to the target following instruction, control the mobile device to enter the target following mode. In the target following mode, collect real-time image data through the camera, and perform target object detection on the real-time image data based on the target image features.

[0058] As some examples, after collecting the target image features, target odor information and initial odor concentration value of the target object, the user can send a target following instruction to the mobile device to make the mobile device enter the target following mode, and then track and follow the target object.

[0059] After the mobile device receives the target following instruction and enters the target following mode, the mobile device can use the camera to capture real-time image data in the current environment, and use the previously collected target image features to perform target object detection on the real-time image data, such as matching the target image features with the features in the real-time image data to detect and identify whether there is a target object in the real-time image data.

[0060] Step 103, in the case where the target object is detected in the real-time image data, determine the position information of the target object, generate the first movement trajectory of the mobile device according to the position information of the target object, and control the mobile device to perform following movement according to the first movement trajectory.

[0061] In some examples, once the target object is successfully detected in the real-time image data, the specific position information of the target object can be further analyzed and determined, such as the coordinates of the target object in the image, the orientation and distance relative to the movable device, etc.; based on this position information, the system can dynamically generate an optimal first movement trajectory, which enables the movable device to efficiently and accurately follow the target object.

[0062] As some examples, the motion control system of the movable device will receive and parse this first movement trajectory, and by adjusting its own motion parameters (such as speed, direction, etc.), enable the movable device to move smoothly along the predetermined trajectory and achieve continuous following of the target object; among them, in this process, the movable device can also continuously collect new real-time image data through the camera and repeat the target object detection and position information determination to ensure the accuracy and continuity of following.

[0063] Step 104, in the case where the target object is not detected in the real-time image data, the real-time temperature value of the temperature sensor, the real-time wind speed value collected by the wind speed sensor, and the real-time odor concentration values of multiple odor sampling points collected by the odor sensor.

[0064] In some examples, if the target object is not detected in the real-time image data, the real-time temperature value of the current environment can be collected by the temperature sensor, the real-time wind speed value of the current environment can be collected by the wind speed sensor, and the real-time odor concentration values of multiple odor sampling points in the current environment can be collected by the odor sensor; among them, the sampling points can be different positions moving outward from the movable device in the current environment. For example, moving east, south, west, and north from the movable device as sampling points respectively, so as to ensure that the odor characteristics of the target object can be effectively captured when the target object is in different orientations, especially when the target object emits a specific odor; at the same time, the setting of multiple sampling points can also improve the accuracy and reliability of odor recognition, and further determine the position and direction of the target object by comparing the odor concentration differences of different sampling points.

[0065] Step 105, determine the distance values between the multiple odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, and the initial odor concentration value, and establish an odor concentration grid map according to the distance values and the real-time odor concentration values; the odor concentration grid map includes multiple grid areas, each grid area has a corresponding odor sampling point, and the real-time odor concentration value of each grid area is the real-time odor concentration value of the corresponding odor sampling point.

[0066] In some embodiments of the present invention, determining the distance values between the plurality of odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, and the initial odor concentration value includes:

[0067] Sub-step 11: Obtain the temperature influence coefficient, the wind speed influence coefficient, the distance attenuation exponent, the odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode.

[0068] In practical applications, the temperature influence coefficient reflects the degree of influence of temperature on odor diffusion; the wind speed influence coefficient reflects the influence of wind speed on the propagation speed and direction of odor; the distance attenuation exponent is used to describe the law of odor concentration attenuation with the increase of distance, and the odor attenuation coefficient takes into account the characteristic that the odor gradually dissipates over time; the duration from the current moment to the moment when entering the target following mode helps to evaluate the change of odor characteristics over time, so as to more accurately locate the target object.

[0069] Sub-step 12: Determine the distance values between the plurality of odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, the initial odor concentration value, the temperature influence coefficient, the wind speed influence coefficient, the distance attenuation exponent, the odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode.

[0070] As some examples, these parameters can be combined and calculated through a comprehensive calculation model to determine the distance values between the plurality of odor sampling points and the movable device. For example, establish a concentration attenuation model of the sensor for special odor: C = (C0 * e^(-k * (1 + aT + bV) * t)) / (r^n); where C is the real-time odor concentration value (ppm), C0 is the initial odor concentration value (ppm), T is the real-time temperature value (°C), V is the real-time wind speed value (m / s), t is the duration from the current moment to the moment when entering the target following mode (s), r is the distance value between the odor sampling point and the movable device (m), k is the odor attenuation coefficient, a is the temperature influence coefficient, b is the wind speed influence coefficient, and n is the distance attenuation exponent.

[0071] In some embodiments of the present invention, the movable device is provided with a plurality of odor sensors located in different directions. Establishing an odor concentration grid map according to the distance value and the real-time odor concentration value includes:

[0072] Sub-step 13: For each odor sampling point, determine the target odor sensor that collects the real-time odor concentration value of the odor sampling point, and determine the position information of the odor sampling point according to the sampling direction information of the target odor sensor and the distance value.

[0073] In some examples, each odor sensor on the mobile device has a specific sampling direction. By comparing the relative positions of the sampling directions of the odor sensors with the odor source, the odor sensor that is closest to the odor source and has the most accurate readings can be selected as the target odor sensor. Then, using the direction information of the target odor sensor and the distance value between the odor sampling point and the mobile device, through geometric calculations, the three-dimensional coordinates of the odor sampling point, that is, the position information, can be accurately deduced.

[0074] Sub-step 14: Taking the mobile device as the center, divide the area within the preset range into multiple grid areas, and establish a corresponding relationship between the position information of the multiple odor sampling points and the position information of the multiple grid areas.

[0075] Among them, the size and shape of each grid area can be set according to actual needs to facilitate subsequent analysis and processing of the odor concentration grid map; by establishing a corresponding relationship, the real-time odor concentration values collected are associated with specific grid areas, thereby generating an odor concentration grid map; through this map, the odor concentration distribution in different areas within the preset range can be intuitively displayed.

[0076] In some examples, after determining the central position of the mobile device, spatial division can be carried out within the preset range (determined based on the device's detection ability, operation requirements, environmental factors, etc.). For example, taking the mobile device as the center, extending 5m forward, backward, left, and right (the collected distance can be adjusted according to the computing power), with a resolution of 10cm (the side length of each grid in the grid area is 10cm), dividing the continuous space into multiple grid areas. Each grid area has its own specific position and boundary, which is convenient for subsequent positioning of odor sampling points and data analysis. Then, according to the position information of the multiple odor sampling points and the position information of the multiple grid areas, a corresponding relationship between the multiple odor sampling points and the multiple grid areas is established, so that subsequent information such as the odor change trend in a certain grid area can be obtained by analyzing the data of multiple sampling points in the grid area.

[0077] Sub-step 15: For each grid area, set the real-time odor concentration value of the corresponding odor sampling point as the real-time odor concentration value of the grid area to establish an odor concentration grid map.

[0078] After establishing the correspondence between multiple odor sampling points and multiple grid regions based on the position information of the multiple odor sampling points and the position information of the multiple grid regions, the real-time odor concentration value of the odor sampling point corresponding to the grid region can be set as the real-time odor concentration value of the grid region, and these grid regions can be combined and displayed according to their spatial positions and corresponding odor concentration values, so as to obtain an odor concentration grid map, which can intuitively reflect the odor concentration distribution of different regions.

[0079] As Figure 2 shown, with the mobile device (ego vehicle) as the center, the area within the preset range is divided into multiple grid regions. For each grid region, the real-time odor concentration value of the corresponding odor sampling point is set as the real-time odor concentration value of the grid region, and an odor concentration grid map is established by representing the real-time odor concentration value through color changes.

[0080] Step 106, according to the real-time odor concentration value of each grid region in the odor concentration grid map, determine multiple target grid regions from the multiple grid regions, generate a second movement trajectory of the mobile device according to the position information of the multiple target grid regions, and control the mobile device to perform following movement according to the second movement trajectory.

[0081] When determining multiple target grid regions, screening can be performed according to factors such as the magnitude, distribution characteristics, or change trend of the real-time odor concentration value. For example, select the grid region with a higher odor concentration value as the target grid region, or select the grid region with a faster changing odor concentration value as the target grid region.

[0082] When generating the second movement trajectory, factors such as the position, quantity, and distribution of the target grid regions, as well as information such as the current position, speed, and direction of the mobile device, can be comprehensively considered, and an optimal movement trajectory can be calculated through a path planning algorithm. Then control the mobile device to perform following movement according to the second movement trajectory. In this way, the mobile device can achieve intelligent following of the target odor, improving the accuracy and efficiency of target following.

[0083] In some embodiments of the present invention, the determining multiple target grid regions from the multiple grid regions includes: for each row of grid regions in the odor concentration grid map, determining the grid region with the largest real-time odor concentration value as the target grid region in each row of grid regions.

[0084] In practical applications, this method can ensure that in each row of grid areas, at least one target grid area is selected for subsequent following movement of the mobile device; by determining the target grid area for each row of grid areas separately, the distribution of the target grid area in the odor concentration grid map can be made more uniform, avoiding target following deviation caused by excessively high or low odor concentration in some areas.

[0085] After determining the target grid area in each row of grid areas, the second movement trajectory of the mobile device can be further generated according to the position information of these target grid areas. When the mobile device follows the movement, it can approach the target odor source more accurately, improving the accuracy and efficiency of target following; at the same time, the determination process of the target grid area is also simplified, improving the practicability and reliability of the target following method.

[0086] In some embodiments of the present invention, the generating the second movement trajectory of the mobile device according to the position information of the multiple target grid areas includes: performing interpolation processing between the position information of two adjacent target grid areas to obtain a plurality of transition position information, and combining the position information of the multiple target grid areas and the transition position information to generate the second movement trajectory of the mobile device.

[0087] In practical applications, this interpolation processing method can make the movement trajectory of the mobile device smoother, avoiding abruptness and instability caused by directly moving from one target grid area to another; by calculating the transition position information between two adjacent target grid areas, the movement trajectory of the mobile device can be made closer to the actual odor concentration distribution; at the same time, combining the position information of multiple target grid areas and the transition position information can generate a complete second movement trajectory, providing clear guidance for the subsequent movement of the mobile device.

[0088] As Figure 3 shown, the mobile device can identify a special odor (odor source), determine the movement trajectory points of the target object according to the odor gradient change in the odor concentration grid map, smooth the trajectory points by the cubic spline interpolation method, and use the pure trajectory tracking method to achieve tracking.

[0089] In specific implementation, first, a sensor array or other odor recognition device is used to detect the odor concentration in the environment in real time, and an odor concentration grid map is constructed. Subsequently, the position of a special odor (i.e., the odor source) is identified on the map, and this position is the initial position of the target object. Then, according to the odor gradient change in the odor concentration grid map, the possible moving direction of the target object is determined. On this basis, a series of trajectory points (i.e., target grid areas) are determined at certain time intervals or spatial intervals on the moving path of the target object; these trajectory points reflect the odor concentration distribution of the target object at different times or different positions.

[0090] In order to make these trajectory points smoother and reduce the fluctuations caused by sampling or detection errors, the cubic spline interpolation method is used to smooth the trajectory points; among them, the cubic spline interpolation method is a method for smooth interpolation between discrete data points, which can ensure that the interpolated curve is continuous at the data points, and its first derivative and second derivative are also continuous, so as to obtain a smooth and realistic curve. By performing cubic spline interpolation on the trajectory points, a smoother moving trajectory of the target object (i.e., the second moving trajectory) can be obtained.

[0091] Finally, a pure trajectory tracking method is used to achieve tracking. The pure trajectory tracking method is a tracking method based on the moving trajectory of the target object, which can calculate the moving strategy and parameters that the tracking device should adopt according to the moving trajectory of the target object, so that the tracking device can accurately follow the movement of the target object. By using the moving trajectory of the target object after cubic spline interpolation as the input of the pure trajectory tracking method, and then calculating the relative position and speed information between the tracking device and the target object, the moving strategy and parameters of the tracking device are determined to achieve accurate tracking of the target object.

[0092] In some embodiments of the present invention, before controlling the movable device to follow the movement according to the second moving trajectory, it further includes: obtaining the kinematic parameters of the movable device and optimizing the second moving trajectory according to the kinematic parameters.

[0093] Among them, the kinematic parameters include but are not limited to the maximum speed, acceleration, steering angle, etc. of the movable device, and these parameters determine the capabilities and limitations of the movable device in actual movement. By optimizing the second moving trajectory, the optimized moving trajectory can be made more in line with the movement characteristics of the movable device, avoiding tracking failure or increased error caused by the device's capacity limitations during actual tracking.

[0094] In some examples, the specific method of the optimization process can be designed according to the type and characteristics of the movable device. For example, for a movable device with a limited steering angle, the limitation on the steering angle can be increased during the optimization process to ensure that the optimized movement trajectory is achievable.

[0095] In some embodiments of the present invention, the movable device is further provided with a sound sensor. Before controlling the movable device to perform following movement according to the second movement trajectory, it further includes:

[0096] Obtaining the audio data collected by the sound sensor for a plurality of audio sampling points; detecting the audio data, and when a target keyword is detected in the audio data, determining the position information of the target audio sampling point corresponding to the audio data; generating a third movement trajectory according to the position information of the target audio sampling point, and calculating the trajectory deviation between the second movement trajectory and the third movement trajectory; when the trajectory deviation is less than a preset deviation value, performing the control to make the movable device perform following movement according to the second movement trajectory; when the trajectory deviation is less than or equal to the preset deviation value, feedback an alarm message.

[0097] Among them, the setting of the sound sensor enables the movable device to have the ability to monitor and analyze the environmental sound when performing the following task. By collecting the audio data of a plurality of audio sampling points, the system can comprehensively capture the surrounding sound information. During the detection process of the audio data, voice recognition technology can be used to perform keyword matching on the collected sound to identify the audio content that matches the preset target keyword.

[0098] Once a target keyword is detected in the audio data, the position information of the target audio sampling point corresponding to the audio data can be determined. Based on the position information of the target audio sampling point, a new third movement trajectory can be generated, and this third movement trajectory can guide the movable device to approach the sound source or a specific target more accurately.

[0099] Subsequently, calculate the trajectory deviation between the second movement trajectory and the third movement trajectory. If the trajectory deviation is less than the preset deviation value, it means that the second movement trajectory is already close enough to the target position. Therefore, the control instruction can be continued to make the movable device perform following movement according to the second movement trajectory. However, if the trajectory deviation is greater than or equal to the preset deviation value, it indicates that there is a large difference between the second movement trajectory and the third movement trajectory, which may be caused by factors such as environmental changes or target movement. In this case, the movable device can feedback an alarm message to remind the operator to pay attention to the trajectory deviation and may need to take further adjustment measures to ensure that the movable device can accurately complete the following task.

[0100] In an embodiment of the present invention, the target odor information and the initial odor concentration value of a target object are collected; target object detection is performed based on real-time image data; when the target object is detected in the real-time image data, a first movement trajectory of a mobile device is generated according to the position information of the target object, and the mobile device is controlled to perform a following movement according to the first movement trajectory; when the target object is not detected in the real-time image data, the position information of a plurality of target grid regions is determined according to the real-time odor concentration value of each grid region in an odor concentration grid map, a second movement trajectory of the mobile device is generated, and the mobile device is controlled to perform a following movement according to the second movement trajectory, thereby realizing generating a movement trajectory of the mobile device according to the situation of target object detection to control the mobile device to perform a following movement according to the movement trajectory, ensuring intelligent, continuous, and accurate tracking of the target object by the mobile device.

[0101] Secondly, the target following method provided by the present invention can be carried out without relying on vision in complex and line-of-sight blocked environments such as darkness, thick fog, and jungles, effectively making up for the deficiencies of humans and mechanical equipment under these conditions. At the scene of an earthquake, fire, or chemical leakage accident, a mobile device (such as an odor tracking robot) can quickly identify and locate survivors or hazardous substances, improving rescue efficiency and safety. Since there are no high-order sensors such as cameras and lidar, privacy can be effectively protected and the occurrence of information leakage can be prevented.

[0102] It should be noted that for method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0103] Some embodiments of the present invention also provide a target following device applied to a mobile device. The mobile device is provided with a camera, a temperature sensor, a wind speed sensor, and an odor sensor. The device is used for:

[0104] In response to a data collection instruction, controlling the mobile device to enter a target image collection mode. In the target image collection mode, the target image features of a target object are collected through the camera, and the mobile device is controlled to enter a target odor collection mode. In the target odor collection mode, the target odor information and the initial odor concentration value of the target object are collected through the odor sensor;

[0105] In response to a target following instruction, control the movable device to enter the target following mode. In the target following mode, collect real-time image data through the camera, and perform target object detection on the real-time image data using the target image features;

[0106] When the target object is detected in the real-time image data, determine the position of the target object. According to the position information of the target object, generate a first movement trajectory of the movable device, and control the movable device to perform following movement according to the first movement trajectory;

[0107] When the target object is not detected in the real-time image data, the real-time temperature value of the temperature sensor, the real-time wind speed value collected by the wind speed sensor, and the real-time odor concentration values of multiple odor sampling points collected by the odor sensor;

[0108] According to the real-time temperature value, the real-time wind speed value, the real-time odor concentration values, and the initial odor concentration value, determine the distance values between the multiple odor sampling points and the movable device, and establish an odor concentration grid map according to the distance values and the real-time odor concentration values; the odor concentration grid map includes multiple grid areas, each grid area has a corresponding odor sampling point, and the real-time odor concentration value of each grid area is the real-time odor concentration value of the corresponding odor sampling point;

[0109] According to the real-time odor concentration values of each grid area in the odor concentration grid map, determine multiple target grid areas from the multiple grid areas. According to the position information of the multiple target grid areas, generate a second movement trajectory of the movable device, and control the movable device to perform following movement according to the second movement trajectory.

[0110] In some embodiments of the present invention, before controlling the movable device to perform following movement according to the second movement trajectory, the device is further configured to:

[0111] Obtain the kinematic parameters of the movable device, and optimize the second movement trajectory according to the kinematic parameters.

[0112] In some embodiments of the present invention, the movable device is provided with multiple odor sensors in different directions, and establishing an odor concentration grid map according to the distance values and the real-time odor concentration values includes:

[0113] For each odor sampling point, determine the target odor sensor that collects the real-time odor concentration value of the odor sampling point, and determine the position information of the odor sampling point according to the sampling direction information of the target odor sensor and the distance value;

[0114] Centering on the movable device, divide the area within a preset range into multiple grid regions, and establish a corresponding relationship between the position information of the multiple odor sampling points and the position information of the multiple grid regions;

[0115] For each grid region, set the real-time odor concentration value of the corresponding odor sampling point as the real-time odor concentration value of the grid region to establish an odor concentration grid map.

[0116] In some embodiments of the present invention, determining multiple target grid regions from the multiple grid regions includes:

[0117] For each row of grid regions in the odor concentration grid map, determine the grid region with the maximum real-time odor concentration value as the target grid region in each row of grid regions.

[0118] In some embodiments of the present invention, generating the second movement trajectory of the movable device according to the position information of the multiple target grid regions includes:

[0119] Perform interpolation processing between the position information of two adjacent target grid regions to obtain multiple transition position information, and combine the position information of the multiple target grid regions and the transition position information to generate the second movement trajectory of the movable device.

[0120] In some embodiments of the present invention, determining the distance values between the multiple odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, and the initial odor concentration value includes:

[0121] Obtain the temperature influence coefficient, the wind speed influence coefficient, the distance attenuation exponent, the odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode;

[0122] Determine the distance values between the multiple odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, the initial odor concentration value, the temperature influence coefficient, the wind speed influence coefficient, the distance attenuation exponent, the odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode.

[0123] In some embodiments of the present invention, the movable device is further provided with a sound sensor, and before controlling the movable device to perform following movement according to the second movement trajectory, the device is further configured to:

[0124] Obtain the audio data collected by the sound sensor for multiple audio sampling points;

[0125] Detect the audio data, and when a target keyword is detected in the audio data, determine the location information of the target audio sampling point corresponding to the audio data;

[0126] Generate a third movement trajectory according to the location information of the target audio sampling point, and calculate the trajectory deviation between the second movement trajectory and the third movement trajectory;

[0127] When the trajectory deviation is less than a preset deviation value, execute controlling the movable device to perform following movement according to the second movement trajectory;

[0128] When the trajectory deviation is less than or equal to the preset deviation value, feedback an alarm message.

[0129] Some embodiments of the present invention further provide an electronic device, including a processor, a memory, and a computer program stored on the memory and capable of running on the processor. When the computer program is executed by the processor, the above method is implemented.

[0130] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor, the above method is implemented.

[0131] Some embodiments of the present invention further provide a computer program product, including a computer program. When the computer program is executed by the processor, the above method is implemented.

[0132] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiments.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for the user to select authorization or rejection.

[0134] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0135] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, an apparatus, or a computer program product. Therefore, the embodiments of the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0136] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0137] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal devices, so that a series of operation steps are executed on the computer or other programmable terminal devices to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable terminal devices provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0139] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0140] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the above element.

[0141] The above provides a detailed introduction to a method, device, electronic device and medium for object following. Specific examples are used in this text to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A method for target following, characterized in that, Applied to a mobile device, the mobile device is provided with a camera, a temperature sensor, a wind speed sensor and an odor sensor, and the method includes: In response to a data collection instruction, control the mobile device to enter a target image acquisition mode. In the target image acquisition mode, collect target image features of a target object through the camera, and control the mobile device to enter a target odor acquisition mode. In the target odor acquisition mode, collect target odor information and an initial odor concentration value of the target object through the odor sensor; In response to a target following instruction, control the mobile device to enter a target following mode. In the target following mode, collect real-time image data through the camera, and perform target object detection on the real-time image data based on the target image features; When the target object is detected in the real-time image data, determine the position information of the target object, generate a first movement trajectory of the mobile device according to the position information of the target object, and control the mobile device to perform following movement according to the first movement trajectory; When the target object is not detected in the real-time image data, obtain the real-time temperature value through the temperature sensor, obtain the real-time wind speed value through the wind speed sensor, and obtain the real-time odor concentration values of multiple odor sampling points collected through the odor sensor; Determine the distance values between the multiple odor sampling points and the mobile device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration values and the initial odor concentration value, and establish an odor concentration grid map according to the distance values and the real-time odor concentration values; the odor concentration grid map includes multiple grid areas, each grid area has a corresponding odor sampling point, and the real-time odor concentration value of each grid area is the real-time odor concentration value of the corresponding odor sampling point; According to the real-time odor concentration values of each grid area in the odor concentration grid map, determine multiple target grid areas from the multiple grid areas, generate a second movement trajectory of the mobile device according to the position information of the multiple target grid areas, and control the mobile device to perform following movement according to the second movement trajectory; 2. The method according to claim 1, characterized in that, Before controlling the mobile device to perform following movement according to the second movement trajectory, it further includes: Obtain the kinematic parameters of the mobile device, and optimize the second movement trajectory according to the kinematic parameters.

3. The method according to claim 1 or 2, characterized in that, The mobile device is provided with multiple odor sensors in different directions, and the establishing of the odor concentration grid map according to the distance values and the real-time odor concentration values includes: For each odor sampling point, determine the target odor sensor that collects the real-time odor concentration value of the odor sampling point, and determine the position information of the odor sampling point according to the sampling direction information of the target odor sensor and the distance value; Centering on the movable device, divide the area within a preset range into multiple grid regions, and establish a corresponding relationship between the position information of the multiple odor sampling points and the position information of the multiple grid regions; For each grid region, set the real-time odor concentration value of the corresponding odor sampling point as the real-time odor concentration value of the grid region to establish an odor concentration grid map.

4. The method according to claim 3, wherein The determining of multiple target grid regions from the multiple grid regions includes: For each row of grid regions in the odor concentration grid map, determine the grid region with the maximum real-time odor concentration value as the target grid region in each row of grid regions.

5. The method according to claim 4, wherein The generating of the second movement trajectory of the movable device according to the position information of the multiple target grid regions includes: Perform interpolation processing between the position information of two adjacent target grid regions to obtain multiple intermediate position information, and combine the position information of the multiple target grid regions and the intermediate position information to generate the second movement trajectory of the movable device.

6. The method according to claim 5, wherein The determining of the distance values between the multiple odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, and the initial odor concentration value includes: Obtain the temperature influence coefficient, the wind speed influence coefficient, the distance attenuation exponent, the odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode; Determine the distance values between the multiple odor sampling points and the movable device according to the real-time temperature value, the real-time wind speed value, the real-time odor concentration value, the initial odor concentration value, the temperature influence coefficient, the wind speed influence coefficient, the distance attenuation exponent, the odor attenuation coefficient, and the duration from the current moment to the moment when entering the target following mode.

7. The method according to claim 6, wherein The movable device is also provided with a sound sensor. Before controlling the movable device to perform following movement according to the second movement trajectory, it further includes: Obtain the audio data collected by the sound sensor for multiple audio sampling points; Detect the audio data, and when a target keyword is detected in the audio data, determine the position information of the target audio sampling point corresponding to the audio data; Generate a third movement trajectory according to the position information of the target audio sampling point, and calculate the trajectory deviation between the second movement trajectory and the third movement trajectory; When the trajectory deviation is less than a preset deviation value, execute the control to make the movable device perform following movement according to the second movement trajectory; When the trajectory deviation is less than or equal to the preset deviation value, feedback an alarm message.

8. A target following device, characterized in that, Applied to a movable device, the movable device is provided with a camera, a temperature sensor, a wind speed sensor, and an odor sensor. The device is used for: In response to a data acquisition instruction, control the movable device to enter a target image acquisition mode. In the target image acquisition mode, collect target image features of a target object through the camera, and control the movable device to enter a target odor acquisition mode. In the target odor acquisition mode, collect target odor information and an initial odor concentration value of the target object through the odor sensor; In response to a target following instruction, control the movable device to enter a target following mode. In the target following mode, collect real-time image data through the camera, and perform target object detection on the real-time image data using the target image features; When the target object is detected in the real-time image data, determine the position of the target object, generate a first movement trajectory of the movable device according to the position information of the target object, and control the movable device to perform following movement according to the first movement trajectory; When the target object is not detected in the real-time image data, obtain the real-time temperature value through the temperature sensor, the real-time wind speed value through the wind speed sensor, and the real-time odor concentration values of multiple odor sampling points collected through the odor sensor; According to the real-time temperature value, the real-time wind speed value, the real-time odor concentration values, and the initial odor concentration value, determine the distance values between the multiple odor sampling points and the movable device, and establish an odor concentration grid map according to the distance values and the real-time odor concentration values; the odor concentration grid map includes multiple grid areas, each grid area has a corresponding odor sampling point, and the real-time odor concentration value of each grid area is the real-time odor concentration value of the corresponding odor sampling point; According to the real-time odor concentration values of each grid area in the odor concentration grid map, determine multiple target grid areas from the multiple grid areas, generate a second movement trajectory of the movable device according to the position information of the multiple target grid areas, and control the movable device to perform following movement according to the second movement trajectory.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 7.

Images