Method and device for determining position, storage medium, and electronic device

Abstract

The present application discloses a method and device for determining a position, a storage medium, and an electronic device, and relates to the field of smart home technology. The method for determining a position includes: collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area; and determining the position of the target object in the target area according to the infrared image of the target area and the audio signal of the target area. The above scheme solves the problem of low accuracy in determining the position of a moving object.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and more specifically, to a method and device for determining a location, a storage medium, and an electronic device. Background Art

[0002] Target positioning technology is widely used for its high precision, low cost and reliable and continuous positioning, such as in the fields of mosquito capture, security monitoring and unmanned driving. Among them, target positioning technology mostly relies on infrared positioning technology, relies on the temperature of the heat source to locate the target, or locates the target by the intensity of the sound signal.

[0003] However, when faced with a target whose temperature does not change much but whose position changes frequently, the accuracy of infrared positioning technology is poor. When faced with a target whose sound intensity is constant but whose position changes frequently and within a small range, target positioning technology cannot accurately locate the target in real time. In addition, relying on sound signals to locate the target is easily affected by environmental noise and weak signals, and the accuracy is not high.

[0004] With respect to the problem of low accuracy in determining the position of a moving object in the related art, no effective solution has been proposed so far.

[0005] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention

[0006] Embodiments of the present application provide a method and device for determining a position, a storage medium, and an electronic device to at least solve the problem of low accuracy in determining the position of a moving object.

[0007] According to one aspect of an embodiment of the present application, a method for determining a position is provided, comprising collecting an infrared image of a target area by an infrared sensor in the target area, and collecting an audio signal of the target area by an audio sensor in the target area; and determining the position of a target object in the target area based on the infrared image of the target area and the audio signal of the target area.

[0008] In an exemplary embodiment, collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area, comprises: when the infrared sensor in the target area includes Z infrared sensors and the audio sensor in the target area includes Z audio sensors, collecting Z target infrared images through the Z infrared sensors, and collecting Z target audio signals through the Z audio sensors, wherein the i-th infrared sensor among the Z infrared sensors and the i-th audio sensor among the Z audio sensors are located at the same position, and Z is an integer greater than or equal to 3; according to the infrared image of the target area and the audio signal of the target area, comprising: determining Z first data through the Z target infrared images, wherein the i-th first data among the Z first data includes the relative position information of the target object relative to the i-th infrared sensor determined according to the i-th target infrared image; determining Z second data through the Z target audio signals, wherein the i-th second data among the Z second data includes the relative position information of the target object relative to the i-th audio sensor determined according to the i-th target audio signal; determining the position of the target object in the target area according to the Z first data and the Z second data, where i is a positive integer less than or equal to Z.

[0009] In an exemplary embodiment, determining the position of the target object in the target area based on the Z first data and the Z second data includes: determining Z target data based on the Z first data and the Z second data, wherein the i-th target data among the Z target data is data determined based on the i-th first data and the i-th second data, and the i-th target data includes relative position information of the target object relative to the i-th infrared sensor or the i-th audio sensor; determining the position of the target object based on the Z position information and the Z target data, wherein the i-th position information among the Z position information is the position information of the i-th infrared sensor or the i-th audio sensor.

[0010] In an exemplary embodiment, Z target data are determined based on the Z first data and the Z second data, including: obtaining the i-th target data among the Z target data in the following manner to obtain the Z target data: fusing the orientation information in the i-th first data and the orientation information in the i-th second data to determine the i-th target orientation information; and / or fusing the distance information in the i-th first data and the distance information in the i-th second data to determine the i-th target distance information; wherein the i-th target data includes the i-th target orientation information and / or the i-th target distance information.

[0011] In an exemplary embodiment, the method further includes: processing the i-th target audio signal among the Z target audio signals in the following manner: determining the frequency information of the target object; determining the i-th feature data corresponding to the frequency information from the i-th target audio signal; or inputting the i-th target audio signal into a target model to obtain the i-th feature data, wherein the target model is used to extract the feature data corresponding to the frequency information from the audio signal; and determining the relative position information of the target object relative to the i-th audio sensor based on the i-th feature data.

[0012] In an exemplary embodiment, the position of a target object in a target area is determined based on an infrared image of the target area and an audio signal of the target area, including: when the infrared image of the target area includes N target infrared images and the audio signal of the target area includes M target audio signals, determining a first position of the target object based on the N target infrared images, and determining a second position of the target object based on the M target audio signals, wherein N and M are both integers greater than or equal to 3; and determining the position of the target object as a middle position between the first position and the second position.

[0013] In an exemplary embodiment, the method further includes: determining P positions of the target object, wherein the p-th position among the P positions is the position of the target object determined based on the infrared image collected by the infrared sensor at the p-th historical moment among P historical moments and the audio signal collected by the audio sensor at the p-th historical moment, the time interval between any two of the P historical moments is less than a preset threshold, and P is an integer greater than or equal to 2; predicting the position of the target object at the current moment based on the P positions, wherein the time interval between a moment among the P historical moments closest to the current moment and the current moment is less than the preset threshold.

[0014] According to another aspect of an embodiment of the present application, a position determination device is provided, comprising: an acquisition module, used to acquire an infrared image of a target area through an infrared sensor in the target area, and to acquire an audio signal of the target area through an audio sensor in the target area; and a determination module, used to determine the position of a target object in the target area based on the infrared image of the target area and the audio signal of the target area.

[0015] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein the computer program is configured to execute the above-mentioned method for determining the position when running.

[0016] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, wherein the processor is configured to execute the above-mentioned method for determining a position through the computer program.

[0017] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and the method for determining the above position when the computer program is executed by a processor.

[0018] Through the present application, since an infrared sensor in the target area is used to collect an infrared image of the target area while an audio sensor in the target area is used to collect an audio signal in the target area, the position of the target object is confirmed in combination with the infrared image and the audio signal, thereby avoiding interference of other environmental factors in the target area with the single data source, resulting in a decrease in the accuracy of the position confirmation result, thereby solving the problem of low accuracy in determining the position of a moving object. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 is a schematic diagram of a hardware environment of a method for determining a position according to an embodiment of the present application;

[0022] Figure 2 is a flow chart of a method for determining a position according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of an optional position determination process according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a position determination module structure according to an embodiment of the present application;

[0025] Figure 5 It is a structural block diagram of another position determination device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to one aspect of an embodiment of the present application, a method for determining a position is provided. The method for determining a position is widely used in smart home (Smart Home), smart home, smart home device ecology, smart house (IntelligenceHouse) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned method for determining a position can be applied to Figure 1 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0029] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0030] In order to solve the above problems, a method for determining a position is provided in this embodiment, including but not limited to being applied to the whole-house intelligent digital control application scenario. Figure 2 is a flow chart of a method for determining a position according to an embodiment of the present application, the flow includes the following steps S202-S204:

[0031] Step S202: collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area;

[0032] Alternatively, if Figure 4 As shown, the infrared sensor module has an infrared detector for collecting infrared images in the target area, and the sound sensor has a microphone for collecting audio signals in the target area.

[0033] Optionally, the system first uses an infrared sensor and an audio sensor set in a target area (e.g., a room or a specific space) to collect data. The infrared sensor can capture the distribution of heat sources in the target area and generate an infrared image. At the same time, the audio sensor is responsible for capturing all audio signals in the target area.

[0034] Step S204: determining the position of the target object in the target area according to the infrared image of the target area and the audio signal of the target area.

[0035] Optionally, the system will combine the heat source location information identified in the infrared image with the audio signal analysis to obtain the location information of the target object, and use data fusion technology (such as weighted averaging, Kalman filtering, etc.) for comprehensive analysis during the analysis process.

[0036] It should be noted that the purpose of integrating multi-source data is to improve the accuracy and robustness of position detection. By comparing the time and space information of infrared images and audio signals, the system can correct false alarms or inaccurate detection problems caused by a single sensor.

[0037] It should be noted that, through the above steps, the limitation of single sensor detection is overcome, and the complementary advantages of infrared sensors and audio sensors are utilized to improve the accuracy and reliability of position detection of target objects.

[0038] In the above steps, the infrared sensor in the target area is used to collect the infrared image of the target area, and the audio sensor in the target area is used to collect the audio signal in the target area. The infrared image and the audio signal are combined to jointly confirm the position of the target object, thereby avoiding interference of other environmental factors in the target area with the single data source, resulting in a decrease in the accuracy of the position confirmation result, thereby solving the problem of low accuracy in determining the position of the mobile object.

[0039] In an exemplary embodiment, determining the position of a target object in the target area based on an infrared image of the target area and an audio signal of the target area includes the following steps: fusing the infrared image and the audio signal to obtain fused data, and determining the position of the target object in the target area based on the fused data.

[0040] Optionally, the original infrared image and audio signal are subjected to noise filtering, feature extraction, and format standardization to ensure that the two types of data can be effectively processed and compared. For example, the audio signal may need to undergo a fast Fourier transform to extract frequency features, while the infrared image may need edge detection or target recognition to locate the heat source. Before fusing the data, the system needs to be time synchronized to ensure that the two data correspond to the same moment or time period during the fusion process. This is usually achieved through a system clock or event trigger mechanism, and spatial alignment is required to map the two data to the same reference coordinate system for subsequent position calculations.

[0041] Optionally, the pre-processed infrared image and audio signal are combined and transformed into fused data through a data fusion algorithm (such as Bayesian fusion, Kalman filtering or neural network). These algorithms can combine the advantages of the two sensors. For example, the infrared image may perform well in low-light environments, while the audio signal can still effectively detect targets in the dark.

[0042] Optionally, the location of the target object is determined based on the fused data, which takes advantage of the complementarity of multi-sensor information. Specific steps may include: Feature analysis: extracting features that are helpful for positioning from the fused data, such as the thermal distribution of the target in the infrared image and the direction of the sound source of the target in the audio signal; Position calculation: using the extracted features and the established positioning model (such as triangulation, maximum likelihood estimation or machine learning model) to calculate the precise location of the target object; Result output: outputting the calculated target location information to the user or system. This information may be used to trigger an alarm, device control or further data analysis.

[0043] It should be noted that through the above-mentioned fusion processing and positioning method, the system can more effectively utilize the comprehensive information of infrared images and audio signals to provide more accurate and robust target positioning results, which has important application value in smart home, security monitoring and biological monitoring and other fields.

[0044] In an exemplary embodiment, collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area can be achieved by the following steps: when the infrared sensor in the target area includes Z infrared sensors and the audio sensor in the target area includes Z audio sensors, collecting Z target infrared images through the Z infrared sensors, and collecting Z target audio signals through the Z audio sensors, wherein the i-th infrared sensor among the Z infrared sensors and the i-th audio sensor among the Z audio sensors are located at the same position, and Z is an integer greater than or equal to 3;

[0045] Optionally, the i-th target infrared image among the Z target infrared images is the target infrared image collected by the i-th infrared sensor among the Z infrared sensors; the i-th target audio signal among the Z target audio signals is the target audio signal collected by the i-th audio sensor among the Z audio sensors.

[0046] It should be noted that in the target area (for example, a room or a closed space), the system is configured with Z infrared sensors and Z audio sensors, where Z is an integer greater than or equal to 3, meaning that at least 3 sets of sensors are used. This multi-sensor layout can capture infrared heat source images and audio signals in the target area from different angles and positions, thereby providing a more comprehensive and multi-perspective data source, which is helpful for subsequent data fusion and position calculation.

[0047] It should be noted that the i-th infrared sensor and the i-th audio sensor in the system are located at the same position in the target area, which means that there are two types of sensors, infrared and audio, at each location. This configuration enables the system to collect infrared images and audio signals at the same time and in the same place, ensuring the temporal and spatial consistency of the two different types of data, which is conducive to subsequent data analysis and fusion.

[0048] In an exemplary embodiment, determining the position of the target object in the target area according to the infrared image of the target area and the audio signal of the target area can be achieved by the following steps S11-S13:

[0049] Step S11: determining Z first data through the Z target infrared images, wherein the i-th first data among the Z first data includes relative position information of the target object relative to the i-th infrared sensor determined according to the i-th target infrared image;

[0050] Optionally, the system identifies the heat source signal of the target object based on the Z target infrared images collected from the Z infrared sensors through image processing technology (such as edge detection, threshold segmentation, feature extraction, etc.). For each infrared image (such as the i-th target infrared image), the system will determine the relative position information of the target object relative to the corresponding infrared sensor (the i-th infrared sensor).

[0051] Optionally, the relative position information includes: coordinates of the target object in the i-th target infrared image, the distance or angle between the target object and the i-th infrared sensor, etc.

[0052] Step S12: determining Z second data through the Z target audio signals, wherein the i-th second data among the Z second data includes relative position information of the target object relative to the i-th audio sensor determined according to the i-th target audio signal;

[0053] Optionally, the system extracts the sound characteristics of the wing vibration of the target object (such as a mosquito) through spectrum analysis (such as Fast Fourier Transform (FFT)). For each audio signal (the i-th target audio signal), the system determines the relative position information of the target object relative to the corresponding audio sensor (the i-th audio sensor) based on acoustic positioning principles such as sound arrival time difference or sound intensity distribution.

[0054] Step S13: determining the position of the target object in the target area according to the Z first data and the Z second data, where i is a positive integer less than or equal to Z.

[0055] Optionally, in step S13, the system will combine all the first data and second data obtained in steps S11 and S12, and comprehensively analyze and process the position information through a data fusion algorithm (such as weighted average, Kalman filtering, least squares method, etc.). The system will compare and fuse each set of first data and second data (i.e., the i-th first data and the i-th second data).

[0056] Optionally, based on the fused data, the system uses a three-dimensional positioning algorithm (such as triangulation, multilateral positioning, or a least squares-based positioning algorithm) to determine the exact position of the target object in the target area. Since each sensor provides the relative position information of the target relative to itself, the system can calculate the three-dimensional coordinates of the target object in space by parsing this information and using mathematical models or geometric relationships.

[0057] It should be noted that the above steps reflect the advantages of multi-sensor collaboration. By combining the analysis of infrared and audio signals, the limitations of a single sensor can be overcome and the accuracy and robustness of the target object position detection can be improved. This technology is particularly suitable for situations that require high-precision positioning, such as mosquito catching devices in smart homes or biological monitoring systems in laboratory environments.

[0058] In an exemplary embodiment, determining the position of the target object in the target area according to the Z first data and the Z second data may be achieved by the following steps S21-S22:

[0059] Step S21: determining Z target data according to the Z first data and the Z second data, wherein the i-th target data among the Z target data is data determined according to the i-th first data and the i-th second data, and the i-th target data includes relative position information of the target object relative to the i-th infrared sensor or the i-th audio sensor;

[0060] Optionally, for each sensor point (i-th infrared sensor and i-th), the system will combine the infrared image analysis result (i-th first data) and the audio signal analysis result (i-th second data) of the point, and combine these two kinds of information through a certain data fusion algorithm (such as weighted average, Kalman filtering, Bayesian estimation, etc.) to determine the precise relative position information of the target object relative to the sensor point. These fused data are Z target data, each of which contains the position information of the target object relative to the corresponding sensor.

[0061] Step S22: determining the position of the target object according to Z pieces of position information and the Z pieces of target data, wherein the i-th piece of position information among the Z pieces of position information is the i-th infrared sensor or the i-th audio sensor position information.

[0062] Optionally, the system will use the Z target data and the known Z sensor position information to finally determine the actual position of the target object in the target area. The Z position information here refers to the exact position coordinates of the Z infrared sensors or audio sensors in space. Since the relative position information of the target object at each sensor point has been obtained in step S21, combined with the absolute position of the sensor, the system can use multi-point positioning technology (such as triangulation positioning, multilateral positioning, etc.) to calculate the three-dimensional coordinates of the target object.

[0063] It should be noted that through the above steps, multi-sensor information fusion and multi-point positioning technology, the limitations of a single sensor such as noise interference and signal blocking are overcome, and the accuracy and robustness of target object position detection are significantly improved. It is suitable for positioning small objects in indoor environments, especially in application scenarios that require high precision and high reliability, such as smart homes, biological monitoring or security systems.

[0064] In an exemplary embodiment, determining Z target data according to the Z first data and the Z second data may be achieved by the following steps: obtaining the i-th target data among the Z target data by the following steps S31 and / or S32 to obtain the Z target data:

[0065] Step S31: fusing the position information in the i-th first data and the position information in the i-th second data to determine the position information of the i-th target;

[0066] Optionally, the system fuses the orientation information in the i-th first data obtained from the infrared image analysis (such as the azimuth and elevation of the target relative to the sensor) and the orientation information in the i-th second data obtained from the audio signal analysis (such as the orientation information based on sound source positioning) for the i-th sensor point (i.e., the location of the i-th infrared sensor and the i-th audio sensor) to obtain the orientation information of the i-th target.

[0067] Alternatively, infrared sensors may perform poorly in dark or nighttime conditions, while audio sensors may not accurately locate in noisy environments. Through fusion processing, the system can dynamically adjust the weight of the two signals in the final position information based on the clarity of the infrared data and the signal-to-noise ratio of the audio data, ensuring better positioning accuracy even under adverse conditions.

[0068] Step S32: fusing the distance information in the i-th first data and the distance information in the i-th second data to determine the i-th target distance information;

[0069] The i-th target data includes the i-th target position information and / or the i-th target distance information.

[0070] Optionally, the system fuses the distance information in the i-th first data obtained from the infrared image analysis (such as the distance between the infrared heat source of the target object and the i-th infrared sensor) and the distance information in the i-th second data obtained from the audio signal analysis (such as the distance between the target object and the i-th audio sensor calculated based on the sound source intensity or the sound time difference) to obtain the i-th target distance information.

[0071] It should be noted that, through the above steps, the orientation information and distance information are fused and processed respectively, which can combine the different characteristics of the infrared sensor and the audio sensor to generate more accurate and reliable i-th target data, and provide high-quality input for the algorithm used to determine the position of the target object (such as the multi-point positioning algorithm), thereby significantly improving the accuracy of positioning the target object in a complex environment.

[0072] In an exemplary embodiment, the method further comprises the following steps: processing the i-th target audio signal among the Z target audio signals through the following steps S41 to S43:

[0073] Step S41: determining the frequency information of the target object;

[0074] Optionally, the system first needs to determine the frequency information that is unique to the target object (such as a mosquito). The sound produced by the vibration of a mosquito's wings will be within a certain frequency range. For example, the frequency of a mosquito's wings may be between 400Hz and 600Hz. Through preliminary research or training, the system can determine this frequency range as frequency information, providing a basis for subsequent audio signal analysis.

[0075] Step S42: determining the i-th feature data corresponding to the frequency information from the i-th target audio signal; or inputting the i-th target audio signal into a target model to obtain the i-th feature data, wherein the target model is used to extract the feature data corresponding to the frequency information from the audio signal;

[0076] Optionally, the system will perform spectral analysis on the audio signal collected by the ith audio sensor (ith target audio signal), focusing on extracting feature data that matches the frequency information of the target object. For example, the audio signal can be converted from the time domain to the frequency domain using techniques such as fast Fourier transform (FFT), and then the part corresponding to the mosquito frequency information can be selected in the frequency domain for further analysis.

[0077] Optionally, the characteristic data includes but is not limited to: the sound intensity, sound time difference or audio frequency difference of the target object within a specific frequency range, and these data will be used in the process of determining the position of the target object.

[0078] Optionally, the system uses a pre-trained target model (such as a deep learning model, a signal processing model) to process the i-th target audio signal. The target model can automatically identify the acoustic features generated by the target object (such as the vibration of mosquito wings) based on the input audio signal and output quantitative indicators related thereto.

[0079] Step S43: Determine the relative position information of the target object with respect to the i-th audio sensor according to the i-th feature data.

[0080] Optionally, the system will use the acoustic positioning principle to determine the relative position information of the target object relative to the i-th audio sensor based on the i-th feature data. For example, the system can calculate the approximate position of the target object by comparing the time difference between multiple audio sensors receiving the sound of the target object; if the sound intensity distribution is used, the system can calculate the position of the target object by analyzing the intensity change of the sound signal in space. Ultimately, the system will output the relative position information of the target object relative to each audio sensor, which will be used together with the infrared sensor data to locate the target object.

[0081] It should be noted that through the processing of the above steps, the system can accurately identify the characteristics of the target object from the audio signal, providing important data support for subsequent comprehensive positioning, especially the positioning accuracy in complex acoustic environments.

[0082] In an exemplary embodiment, determining the position of the target object in the target area according to the infrared image of the target area and the audio signal of the target area can be achieved by the following steps S51-S52:

[0083] Step S51: when the infrared image of the target area includes N target infrared images and the audio signal of the target area includes M target audio signals, determining a first position of the target object according to the N target infrared images, and determining a second position of the target object according to the M target audio signals, wherein N and M are both integers greater than or equal to 3;

[0084] Optionally, the system first uses the infrared images in the target area collected by N (N>=3) infrared sensors to perform image processing and analysis, determine the position information of the target object (such as a mosquito) in the infrared image, and thus calculate the first position of the target object relative to these infrared sensors. At the same time, the system also uses the audio signals in the target area collected by M (M>=3) audio sensors to extract acoustic features, and determines the second position of the target object relative to these audio sensors through a sound source localization algorithm.

[0085] It should be noted that N and M need to be greater than or equal to 3 because multi-point positioning technology (such as triangulation positioning) requires at least 3 sensor points to determine the position of a target in three-dimensional space. Through the collaborative work of multiple sensors, the system can obtain multiple angles and position information of the target, which helps to improve the accuracy and robustness of positioning. Even if a single sensor data is wrong, the redundancy and complementarity of multi-sensor data can ensure the reliability of the final positioning result.

[0086] Step S52: determining the position of the target object by taking the middle position between the first position and the second position.

[0087] Optionally, based on the first position and the second position, the position of the target object is determined by weighted averaging or fusion processing, wherein the position of the target object is a position between the first position and the second position, wherein the weights of the first position and the second position can be dynamically adjusted according to the performance of various sensors, environmental conditions and the quality of real-time data. For example, if the ambient light conditions are poor, the performance of the infrared sensor may decrease, and the system will give a higher weight to audio signal positioning; conversely, if the noise environment is serious, the system may rely more on infrared image positioning.

[0088] It should be noted that by using the intermediate position as the final positioning result, the system can combine the different advantages of infrared sensors and audio sensors to provide a more accurate and reliable target positioning. This method not only enhances the detection and positioning capabilities of the target object, but also improves the adaptability and flexibility of the system, enabling it to work stably in a variety of different environments.

[0089] In an exemplary embodiment, the method further includes the following steps S61-S62:

[0090] Step S61: determining P positions of the target object, wherein the p-th position among the P positions is the position of the target object determined based on the infrared image collected by the infrared sensor at the p-th historical moment among the P historical moments and the audio signal collected by the audio sensor at the p-th historical moment, the time interval between any two historical moments among the P historical moments is less than a preset threshold, and P is an integer greater than or equal to 2;

[0091] Optionally, the system analyzes the position of the target object in the historical time series, and the P positions include the location records of the target object at P historical moments, and at least two sensor data, infrared image and audio signal, are used at each moment to determine the position of the target. The preset threshold of the time interval is to ensure that the position information of these historical moments is close and continuous enough to be used for subsequent prediction analysis. Selecting P as an integer greater than or equal to 2 means that at least two historical moments of position data are required to establish a time series model.

[0092] Step S62: predicting the position of the target object at the current moment based on the P positions, wherein the time interval between a moment among the P historical moments closest to the current moment and the current moment is less than the preset threshold.

[0093] Optionally, the system uses the time series of historical location information, combined with the infrared image and audio signal at the current moment, and uses a time series prediction algorithm, such as Kalman filtering, particle filtering, or autoregressive integrated moving average model to predict the precise location of the target object at the current moment. The time series prediction algorithm can take into account the position change trend of the target object in the past moments, combined with the instant feedback of the current sensor data, to give a more accurate location prediction. In particular, the system will focus on analyzing the historical location information closest to the current moment, because this part of information is most relevant to the current moment and can more accurately reflect the instant movement state and position of the target object.

[0094] It should be noted that by using time series analysis, the system can effectively handle instantaneous sensor data anomalies or environmental interference, and ensure the continuity and stability of positioning results through the smoothing effect of historical data.

[0095] It should be noted that through the above steps, combined with the dynamic prediction of historical location information and the real-time positioning of current sensor data, the system can more accurately track the movement trajectory of the target object, and provide stable and accurate location information even when the target object moves quickly or the environmental conditions change suddenly. This is crucial for application scenarios that require real-time monitoring and tracking of moving targets (such as smart security, smart home, etc.).

[0096] It should be noted that, through the implementation of steps S61 and S62, the system can not only respond to current sensor data instantly, but also use past information for prediction, thereby providing a fast and accurate target positioning solution, significantly improving the performance and applicability of the positioning system.

[0097] In an exemplary embodiment, after determining the position of the target object in the target area based on the infrared image of the target area and the audio signal of the target area, the method further includes: sending the position of the target object through a display screen, and sending an alarm message through an alarm device, wherein the alarm information is used to indicate the presence of the target object in the target area.

[0098] Optionally, the alarm device includes but is not limited to: a light emitting diode (LED) and a buzzer.

[0099] Obviously, the above-described embodiments are only embodiments of a part of the present invention, rather than all embodiments. In order to better understand the above method, the above process is described below in conjunction with embodiments, but it is not intended to limit the technical solutions of the embodiments of the present invention, specifically:

[0100] Figure 3 (Step S301 to Step S307) illustrates an optional process for determining a position, when the target object is a mosquito, specifically as follows:

[0101] 1. Sound signal (i.e. the above-mentioned audio signal) acquisition and processing:

[0102] like Figure 4 The sound sensor module shown in the figure can use a high-sensitivity microphone to capture the high-frequency sound signal generated by the vibration of the mosquito's wings, and extract the sound characteristics of specific frequencies through fast Fourier transform (FFT);

[0103] 2. Infrared signal (i.e. the above-mentioned infrared image) acquisition and processing:

[0104] like Figure 4 The infrared sensor module shown in the figure can use a high-sensitivity infrared detector to detect the infrared heat signal of mosquitoes, remove the influence of background heat sources, and identify moving targets;

[0105] 3. Data Fusion:

[0106] like Figure 4 The data fusion analysis module shown in the figure, in which the data processing unit can perform time synchronization and spatial registration on the sound signal and the infrared signal, and fuse the multi-sensor data through algorithms such as weighted averaging and Kalman filtering;

[0107] 4. Position calculation:

[0108] Use the fused data to calculate the exact location of the mosquito through algorithms such as triangulation;

[0109] 5. Prompt the user:

[0110] When a mosquito is detected, the user is notified via an LED indicator, buzzer or display.

[0111] To be clear, the following is a simple pseudocode example:

[0112] import numpy as np;

[0113] from scipy.fftpack import fft;

[0114] def process_sound_signal(signal,sample_rate):#Perform fast Fourier transform;

[0115] N = len(signal);

[0116] yf = fft(signal);

[0117] xf=np.linspace(0.0,sample_rate / 2.0,N / / 2);

[0118] #Only consider the positive frequency part;

[0119] sound_spectrum=2.0 / N*np.abs(yf[:N / / 2]);

[0120] #Detect peaks within a specific frequency range;

[0121] mosquito_frequency_range = (400,600) #Assume that the frequency of mosquito wing vibration is between 400-600Hz;

[0122] indices=(xf>mosquito_frequency_range[0])&(xf <mosquito_frequency_range[1]);

[0123] if np.max(sound_spectrum[indices])>threshold:#threshold is the preset judgment threshold;

[0124] return True;

[0125] return False;

[0126] def process_ir_signal(ir_data):

[0127] #Detect features in infrared signals;

[0128] #Here it is assumed that ir_data is the pre-processed infrared sensor data;

[0129] if np.max(ir_data)>ir_threshold:#ir_threshold is the preset judgment threshold;

[0130] return True;

[0131] return False;

[0132] def detect_mosquito(sound_signal,ir_data,sample_rate):

[0133] sound_detected=process_sound_signal(sound_signal,sample_rate);

[0134] ir_detected=process_ir_signal(ir_data);

[0135] if sound_detected and ir_detected:

[0136] #Prompt the user through LED indicator, buzzer or display;

[0137] activate_user_alert();

[0138] return True;

[0139] return False;

[0140] #Example call:

[0141] mosquito_position=detect_mosquito(sound_signal,ir_data,sample_rate);

[0142] print("Mosquito detected:",mosquito_position).

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0144] In this embodiment, a device for determining a position is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0145] Figure 5 is a structural block diagram of a position determination device according to an embodiment of the present application, the device comprising:

[0146] A collection module 52, used to collect an infrared image of the target area through an infrared sensor in the target area, and to collect an audio signal of the target area through an audio sensor in the target area;

[0147] The determination module 54 is configured to determine the position of the target object in the target area according to the infrared image of the target area and the audio signal of the target area.

[0148] The above-mentioned device uses the infrared sensor in the target area to collect the infrared image of the target area while using the audio sensor in the target area to collect the audio signal in the target area. The infrared image and the audio signal are combined to jointly confirm the position of the target object, thereby avoiding other environmental factors in the target area from interfering with the single data source, resulting in a decrease in the accuracy of the position confirmation result, thereby solving the problem of low accuracy in determining the position of the mobile object.

[0149] In an exemplary embodiment, the acquisition module 52 is further used to acquire Z target infrared images through the Z infrared sensors and acquire Z target audio signals through the Z audio sensors when the infrared sensors in the target area include Z infrared sensors and the audio sensors in the target area include Z audio sensors, wherein the i-th infrared sensor among the Z infrared sensors and the i-th audio sensor among the Z audio sensors are located at the same position, and Z is an integer greater than or equal to 3; the determination module 54 is further used to determine Z first data through the Z target infrared images, wherein the i-th first data among the Z first data includes the relative position information of the target object relative to the i-th infrared sensor determined according to the i-th target infrared image; determine Z second data through the Z target audio signals, wherein the i-th second data among the Z second data includes the relative position information of the target object relative to the i-th audio sensor determined according to the i-th target audio signal; determine the position of the target object in the target area according to the Z first data and the Z second data, and i is a positive integer less than or equal to Z.

[0150] In an exemplary embodiment, the determination module 54 is further used to determine Z target data based on the Z first data and the Z second data, wherein the i-th target data among the Z target data is data determined based on the i-th first data and the i-th second data, and the i-th target data includes relative position information of the target object relative to the i-th infrared sensor or the i-th audio sensor; determine the position of the target object based on the Z position information and the Z target data, wherein the i-th position information among the Z position information is the position information of the i-th infrared sensor or the i-th audio sensor.

[0151] In an exemplary embodiment, the determination module 54 is also used to obtain the i-th target data among the Z target data in the following manner to obtain the Z target data: fusing the azimuth information in the i-th first data and the azimuth information in the i-th second data to determine the i-th target azimuth information; and / or fusing the distance information in the i-th first data and the distance information in the i-th second data to determine the i-th target distance information; wherein the i-th target data includes the i-th target azimuth information and / or the i-th target distance information.

[0152] In an exemplary embodiment, the above-mentioned device also includes: a processing module, which is used to process the i-th target audio signal among the Z target audio signals in the following manner: determining the frequency information of the target object; determining the i-th feature data corresponding to the frequency information from the i-th target audio signal; or inputting the i-th target audio signal into a target model to obtain the i-th feature data, wherein the target model is used to extract the feature data corresponding to the frequency information from the audio signal; and determining the relative position information of the target object relative to the i-th audio sensor based on the i-th feature data.

[0153] In an exemplary embodiment, the determination module 54 is also used to determine the first position of the target object according to the N target infrared images and the second position of the target object according to the M target audio signals when the infrared image of the target area includes N target infrared images and the audio signal of the target area includes M target audio signals, wherein N and M are both integers greater than or equal to 3; and determine the position of the target object as the middle position between the first position and the second position.

[0154] In an exemplary embodiment, the determination module 54 is also used to determine P positions of the target object, wherein the p-th position among the P positions is the position of the target object determined based on the infrared image collected by the infrared sensor at the p-th historical moment among P historical moments and the audio signal collected by the audio sensor at the p-th historical moment, and the time interval between any two of the P historical moments is less than a preset threshold, and P is an integer greater than or equal to 2; the above-mentioned device also includes: a prediction module, used to predict the position of the target object at the current moment based on the P positions, wherein the time interval between the moment closest to the current moment among the P historical moments and the current moment is less than the preset threshold.

[0155] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein the computer program is configured to execute the above-mentioned method for determining a position when running.

[0156] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0157] S1, collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area;

[0158] S2, determining a position of a target object in the target area according to the infrared image of the target area and the audio signal of the target area.

[0159] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0160] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0161] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, wherein the processor is configured to execute the above-mentioned method for determining a position through the computer program.

[0162] Optionally, in this embodiment, the electronic device may be configured to perform the following steps through a computer program:

[0163] S1, collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area;

[0164] S2, determining a position of a target object in the target area according to the infrared image of the target area and the audio signal of the target area.

[0165] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0166] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and the method for determining the above position when the computer program is executed by a processor.

[0167] Optionally, in this embodiment, the computer program product may be configured to perform the following steps through a computer program:

[0168] S1, collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area;

[0169] S2, determining a position of a target object in the target area according to the infrared image of the target area and the audio signal of the target area.

[0170] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0171] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0172] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining a position, characterized in that: include: Collecting an infrared image of the target area through an infrared sensor in the target area, and collecting an audio signal of the target area through an audio sensor in the target area; The position of the target object in the target area is determined according to the infrared image of the target area and the audio signal of the target area.

2. The method according to claim 1, characterized in that The infrared image of the target area is collected by the infrared sensor in the target area, and the audio signal of the target area is collected by the audio sensor in the target area, including: when the infrared sensor in the target area includes Z infrared sensors and the audio sensor in the target area includes Z audio sensors, Z target infrared images are collected by the Z infrared sensors, and Z target audio signals are collected by the Z audio sensors, wherein the i-th infrared sensor in the Z infrared sensors and the i-th audio sensor in the Z audio sensors are located at the same position, and Z is an integer greater than or equal to 3; Determining the position of a target object in the target area according to the infrared image of the target area and the audio signal of the target area, comprising: determining Z first data through the Z target infrared images, wherein the i-th first data among the Z first data includes the relative position information of the target object relative to the i-th infrared sensor determined according to the i-th target infrared image; determining Z second data through the Z target audio signals, wherein the i-th second data among the Z second data includes the relative position information of the target object relative to the i-th audio sensor determined according to the i-th target audio signal; determining the position of the target object in the target area according to the Z first data and the Z second data, i being a positive integer less than or equal to Z.

3. The method according to claim 2, characterized in that Determining the position of the target object in the target area according to the Z first data and the Z second data includes: Determine Z target data according to the Z first data and the Z second data, wherein the i-th target data among the Z target data is data determined according to the i-th first data and the i-th second data, and the i-th target data includes relative position information of the target object relative to the i-th infrared sensor or the i-th audio sensor; The position of the target object is determined according to Z pieces of position information and the Z pieces of target data, wherein the i-th piece of position information among the Z pieces of position information is the i-th infrared sensor or the i-th audio sensor position information.

4. The method according to claim 3, characterized in that Determining Z target data according to the Z first data and the Z second data includes: The i-th target data among the Z target data is obtained by the following method to obtain the Z target data: fusing the position information in the i-th first data and the position information in the i-th second data to determine the i-th target position information; and / or Fusing the distance information in the i-th first data and the distance information in the i-th second data to determine the i-th target distance information; The i-th target data includes the i-th target position information and / or the i-th target distance information.

5. The method according to claim 2, characterized in that: The method further comprises: The i-th target audio signal among the Z target audio signals is processed in the following manner: determining the frequency information of the target object; determining the i-th feature data corresponding to the frequency information from the i-th target audio signal; or inputting the i-th target audio signal into a target model to obtain the i-th feature data, wherein the target model is used to extract the feature data corresponding to the frequency information from the audio signal; The relative position information of the target object relative to the i-th audio sensor is determined according to the i-th feature data.

6. The method according to claim 1, characterized in that Determining a position of a target object in the target area according to the infrared image of the target area and the audio signal of the target area includes: In a case where the infrared image of the target area includes N target infrared images and the audio signal of the target area includes M target audio signals, a first position of the target object is determined according to the N target infrared images, and a second position of the target object is determined according to the M target audio signals, wherein N and M are both integers greater than or equal to 3; The position of the target object is determined as an intermediate position between the first position and the second position.

7. The method according to claim 1, characterized in that The method further comprises: Determine P positions of the target object, wherein the p-th position among the P positions is the position of the target object determined based on the infrared image collected by the infrared sensor at the p-th historical moment among the P historical moments and the audio signal collected by the audio sensor at the p-th historical moment, the time interval between any two historical moments among the P historical moments is less than a preset threshold, and P is an integer greater than or equal to 2; The position of the target object at a current moment is predicted based on the P positions, wherein a time interval between a moment among the P historical moments closest to the current moment and the current moment is less than the preset threshold.

8. A device for determining a position, characterized in that: include: An acquisition module, used to acquire an infrared image of the target area through an infrared sensor in the target area, and to acquire an audio signal of the target area through an audio sensor in the target area; The determination module is used to determine the position of the target object in the target area according to the infrared image of the target area and the audio signal of the target area.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images