An obstacle detection method, apparatus, electronic device, and storage medium

Abstract

This invention provides an obstacle detection method, apparatus, electronic device, and storage medium. By acquiring a first image of a detection area, and upon determining the presence of an obstacle in the detection area, a second image corresponding to the obstacle is extracted from the first image. The feature information of the obstacle is determined based on the positional information of the second image within the first image. This solution can determine the obstacle's own feature information based on the image corresponding to the obstacle. Since the obstacle's feature information reflects its characteristics, obstacles can be distinguished based on the acquired feature information, thus solving the current technical problem of being unable to distinguish obstacles.

Description

Technical Field

[0001] This application relates to the field of obstacle detection, and more particularly to an obstacle detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] Detecting obstacles during air conditioner operation helps adjust the airflow mode based on the obstacles, thereby achieving better cooling or heating effects.

[0003] Different types of obstacles have different effects on the airflow of air conditioners. Therefore, to achieve better airflow performance, different airflow adjustment strategies should be adopted for different types of obstacles. However, currently, air conditioners typically use pyroelectric or microwave radar methods to detect the location of obstacles, which cannot distinguish between obstacles based on their location alone. Therefore, there is an urgent need for a new obstacle detection method to differentiate between obstacles. Summary of the Invention

[0004] To address the technical problem of being unable to distinguish obstacles, this application provides an obstacle detection method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, embodiments of this application provide an obstacle detection method, including:

[0006] Acquire the first image of the detection area;

[0007] If an obstacle is detected within the detection area, a second image corresponding to the obstacle is extracted from the first image;

[0008] Based on the position information of the second image in the first image, the feature information of the obstacle is determined.

[0009] As one possible implementation, the feature information includes the ground projection distance between the obstacle and the image acquisition device corresponding to the first image, and the step of determining the feature information of the obstacle based on the position information of the second image in the first image includes:

[0010] Determine the number of first pixel rows that differ between the bottom edge of the second image and the bottom edge of the first image;

[0011] Based on the first pixel row number, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device corresponding to the first image, determine the first included angle between the lower edge of the obstacle and the lower edge of the frame of the image acquisition device;

[0012] Based on the included angle between the image acquisition device and the mounting surface and the visible pitch angle of the image acquisition device, determine the second included angle between the lower edge of the frame of the image acquisition device and the mounting surface;

[0013] Based on the first intersection angle, the second included angle, and the distance between the image acquisition device and the ground, the ground projection distance between the obstacle and the image acquisition device is determined.

[0014] As one possible implementation, based on the first angle, the second included angle, and the distance between the image acquisition device and the ground, the ground projection distance between the obstacle and the image acquisition device is determined according to the following formula:

[0015] d1 = h * tan(δ + ε)

[0016] δ=(π / 2–(β / 2+α))

[0017] ε=β*(L1 / Row)

[0018] Where d1 represents the ground projection distance between the obstacle and the image acquisition device, h represents the distance between the image acquisition device and the ground, δ represents the second included angle, β represents the visible pitch angle of the image acquisition device, α represents the included angle between the image acquisition device and the mounting surface, ε represents the first included angle, L1 represents the first pixel row number, and Row represents the total number of pixel rows of the first image.

[0019] As one possible implementation, the feature information further includes the height of the obstacle, and the step of determining the feature information of the obstacle based on the position information of the second image in the first image includes:

[0020] Determine the total number of rows of pixels in the second image;

[0021] The longitudinal proportion angle of the obstacle is determined based on the total number of pixel rows in the second image, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device.

[0022] The ground projection distance between the upper edge of the obstacle and the image acquisition device is determined based on the first included angle, the second included angle, the longitudinal proportion angle, and the distance between the image acquisition device and the ground.

[0023] The height of the obstacle is determined based on the first included angle, the second included angle, the ground projection distance between the upper edge of the obstacle and the image acquisition device, and the ground projection distance between the obstacle and the image acquisition device.

[0024] As one possible implementation, the height of the obstacle is determined based on the first included angle, the second included angle, the ground projection distance between the upper edge of the obstacle and the image acquisition device, and the ground projection distance between the obstacle and the image acquisition device, including:

[0025] h2=(d2-d1) / tan(δ+ε+ε2)

[0026] ε2=β*(L2 / Row)

[0027] d² = h * tan(δ + ε + ε²)

[0028] Where h2 represents the height of the obstacle, d2 represents the ground projection distance between the upper edge of the obstacle and the image acquisition device, ε2 represents the longitudinal proportion angle, and L2 represents the total number of pixel rows in the second image.

[0029] As one possible implementation, the feature information further includes the width of the obstacle, and determining the feature information of the obstacle based on the position information of the second image in the first image includes:

[0030] Determine the total number of columns of pixels in the second image;

[0031] The lateral proportion angle of the obstacle is determined based on the total number of pixel columns in the second image, the total number of pixel columns in the first image, and the visible horizontal angle of the image acquisition device.

[0032] The width of the obstacle is determined based on the lateral proportion angle and the ground projection distance between the obstacle and the image acquisition device.

[0033] As one possible implementation, the width of the obstacle is determined according to the following formula, based on the lateral proportion angle and the ground projection distance between the obstacle and the image acquisition device:

[0034] W = 2 * d1 * tan(θ / 2)

[0035] θ = ρ * (L3 / Column)

[0036] Wherein, W represents the width of the obstacle, θ represents the lateral proportion angle, ρ represents the horizontal viewing angle of the image acquisition device, L3 represents the total number of pixel columns in the second image, and Column represents the total number of pixel columns in the first image.

[0037] As one possible implementation, the method further includes:

[0038] The aspect ratio of the obstacle is determined based on its width and height.

[0039] Determine whether the obstacle has moved;

[0040] Determine the surface temperature of the obstacle;

[0041] If the aspect ratio is less than a preset first threshold, the obstacle moves, and the surface temperature is within a preset temperature range, then the obstacle is determined to be a human; otherwise, the obstacle is determined to be a non-human object.

[0042] As one possible implementation, the method further includes:

[0043] If the obstacle is determined to be a non-human object, then determine whether the shape of the second image is a preset shape;

[0044] If the shape of the second image is determined to be a preset shape, then according to the preset correspondence between the aspect ratio of an object and the object type, the target type corresponding to the aspect ratio of the obstacle is determined, and the target type is determined to be the type of the obstacle.

[0045] As one possible implementation, the method further includes:

[0046] The feature information is input into a pre-trained object classification model, which then outputs the type of obstacle.

[0047] Secondly, embodiments of this application also provide an obstacle detection device, comprising:

[0048] The acquisition module is used to acquire the first image of the detection area;

[0049] The extraction module is used to extract a second image corresponding to an obstacle from the first image when it is determined that there is an obstacle in the detection area;

[0050] The determining module is used to determine the feature information of the obstacle based on the position information of the second image in the first image.

[0051] Thirdly, embodiments of this application also provide an electronic device, including: a processor and a memory, wherein the processor is configured to execute a data processing program stored in the memory to implement the obstacle detection method described in the first aspect.

[0052] Fourthly, embodiments of this application also provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the obstacle detection method described in the first aspect.

[0053] The technical solutions provided in this application have the following advantages compared with the prior art:

[0054] This invention provides an obstacle detection method. By acquiring a first image of a detection area, and when an obstacle exists in the detection area, a second image corresponding to the obstacle is extracted from the first image. The feature information of the obstacle is determined based on the position information of the second image in the first image. This solution can determine the obstacle's own feature information based on the image corresponding to the obstacle. Since the obstacle's feature information reflects its characteristics, obstacles can be distinguished based on the acquired feature information, thus solving the current technical problem of being unable to distinguish obstacles. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating an obstacle detection method provided in this application embodiment;

[0058] Figure 2 A flowchart illustrating a method for determining the distance between an obstacle and an image acquisition device, as provided in this application embodiment;

[0059] Figure 3 A first image provided for an embodiment of this application;

[0060] Figure 4 An imaging principle diagram provided for an embodiment of this application;

[0061] Figure 5 A flowchart illustrating a method for determining the height of an obstacle, as provided in this application embodiment;

[0062] Figure 6 A flowchart illustrating a method for determining the width of an obstacle, as provided in an embodiment of this application;

[0063] Figure 7 A flowchart illustrating another obstacle detection method provided in this application embodiment;

[0064] Figure 8 A flowchart for determining whether there is an obstacle in a detection area is provided in an embodiment of this application;

[0065] Figure 9 A block diagram of an obstacle detection device provided in an embodiment of this application;

[0066] Figure 10 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] Figure 1 This application provides a flowchart of an obstacle detection method, which can be applied to devices such as air conditioners. Figure 1 As shown, the method may include the following steps:

[0069] S11. Obtain the first image of the detection area.

[0070] As an example, the first image is an image of the detection area acquired by an image acquisition device, which may be an optical camera or an infrared array sensor, etc.

[0071] The detection area can be set according to actual needs. For example, if this obstacle detection method is applied to an air conditioner, the detection area can be the air supply area of ​​the air conditioner.

[0072] S12. If it is determined that there is an obstacle in the detection area, extract the second image corresponding to the obstacle from the first image.

[0073] As an optional implementation, if the first image of the detection area is a temperature distribution image, the presence of obstacles in the detection area can be determined based on the temperature values ​​corresponding to the pixels in the first image. The method of determining whether there are obstacles in the detection area based on the temperature values ​​of the pixels will be described below, and will not be elaborated here.

[0074] As another alternative implementation, if the first image is an image captured by an optical camera, existing image recognition technology can be used to determine whether there are obstacles in the detection area.

[0075] As an example, after determining that an obstacle exists in the detection area, image segmentation technology can be used to extract the second image corresponding to the obstacle from the first image. Specifically, this can include the following steps:

[0076] Step 1: Convert the first image to grayscale.

[0077] Step 2: Determine the histogram corresponding to the grayscale image.

[0078] Step 3: Smooth and analyze the histogram, and find the valley temperature boundary point using the bimodal method or median method to obtain the segmentation threshold z.

[0079] A good segmentation threshold should correspond to the minimum value between two peaks in the histogram. For example, if the first image is a temperature distribution map, and the temperature in the temperature distribution map is relatively concentrated, for example, mainly concentrated between 22℃ and 40℃, it is not easy to analyze. Therefore, before step 3, the grayscale image can be equalized at levels 0 to 255 to improve the contrast. Then, step 3 is used to smooth and analyze the equalized temperature distribution histogram, and the dividing point threshold z is found by the bimodal method.

[0080] Step 4: Binarize the grayscale image according to the segmentation threshold z to obtain the shape map of the obstacle.

[0081] As an optional implementation, the obtained shape map can be directly used as the second image corresponding to the obstacle.

[0082] As another optional implementation, the region corresponding to the shape map can be used as the target region, and the image corresponding to the target region in the first image can be extracted as the second image corresponding to the obstacle.

[0083] S13. Determine the feature information of the obstacle based on the position information of the second image in the first image.

[0084] As an example, the feature information of the obstacle may include one or more of the following features: the ground projection distance between the obstacle and the image acquisition device corresponding to the first image, the height of the obstacle, the width of the obstacle, etc.

[0085] As an example, if the feature information includes the ground projection distance between the obstacle and the image acquisition device corresponding to the first image, then as follows: Figure 2 As shown, determining the feature information of the obstacle based on the position information of the second image in the first image may include:

[0086] S21. Determine the number of first pixel rows that differ between the lower edge of the second image and the lower edge of the first image.

[0087] The lower edge of the second image is the edge in the second image that is closest to the lower edge of the first image.

[0088] The number of pixel rows can be understood as the number of pixels in a vertical direction. The first pixel row number refers to the number of pixels that are vertically different between the bottom edge of the second image and the bottom edge of the first image. For ease of understanding, the first pixel row number is explained as follows: Take any pixel in the bottom edge of the second image as the first pixel. Draw a line perpendicular to the bottom edge of the first image through the first pixel. The pixel corresponding to the intersection of this vertical line and the bottom edge of the first image is taken as the second pixel. The number of pixels between the first pixel and the second pixel in the vertical line is the number of pixels that are vertically different between the bottom edges of the second and first images, which is also the first pixel row number. The above description is only for ease of understanding of the first pixel row number and does not constitute a limitation on the method of determining the first pixel row number.

[0089] For example Figure 3 As shown, Figure 3 This is a schematic diagram of the first image, where the dark area represents the second image, and L1 represents the first pixel row number.

[0090] S22. Based on the first pixel row number, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device corresponding to the first image, determine the first included angle between the lower edge of the obstacle and the lower edge of the frame of the image acquisition device.

[0091] The total number of rows and columns of pixels in an image can be determined based on the image's resolution. The total number of rows is the image's vertical resolution, and the total number of columns is the image's horizontal resolution. For example, if the image resolution is 800*600, then the total number of rows is 600 and the total number of columns is 800. Therefore, the total number of rows in the first image is its vertical resolution. The resolution of the first image is an attribute value and can be directly obtained.

[0092] The visible pitch angle of the image acquisition device is a parameter of the image acquisition device itself. It can be stored in a preset location in advance and then directly obtained from the preset location when S22 is executed. Alternatively, it can be connected to the network when S22 is executed and then search for the field of view of the image acquisition device online based on the model of the image acquisition device. The field of view includes the visible pitch angle and the visible horizontal angle.

[0093] As an example, the first included angle can be calculated using the following formula:

[0094] ε=β*(L1 / Row)

[0095] Where ε represents the first included angle, β represents the visible pitch angle of the image acquisition device, L1 represents the first pixel row number, and Row represents the total number of pixel rows in the first image.

[0096] S23. Determine the second angle between the lower edge of the frame of the image acquisition device and the mounting surface based on the included angle between the image acquisition device and the mounting surface and the visible pitch angle of the image acquisition device.

[0097] As an example, the mounting surface is the plane on which the image acquisition device is installed. The angle between the image acquisition device and the mounting surface is set according to actual needs during the installation of the image acquisition device. Specifically, after the image acquisition device is installed, the angle between the image acquisition device and the mounting surface can be stored in a preset position. When executing S23, the angle between the image acquisition device and the mounting surface can be directly obtained from the preset position.

[0098] As an example, the second included angle can be calculated using the following formula:

[0099] δ=(π / 2–(β / 2+α))

[0100] Where δ represents the second included angle, and α represents the included angle between the image acquisition device and the mounting surface, for details please refer to Figure 4 .

[0101] S24. Based on the first intersection angle, the second included angle, and the distance between the image acquisition device and the ground, determine the ground projection distance between the obstacle and the image acquisition device.

[0102] The distance between the image acquisition device and the ground is the installation height of the image acquisition device, which is also set according to actual needs. Specifically, after the image acquisition device is installed, the installation height of the image acquisition device can be stored in a preset position. When executing S24, the installation height of the image acquisition device can be directly obtained from the preset position.

[0103] As an example, the ground projection distance between the obstacle and the image acquisition device can be determined according to the following formula:

[0104] d1 = h * tan(δ + ε)

[0105] Where d1 represents the ground projection distance between the obstacle and the image acquisition device, and h represents the distance between the image acquisition device and the ground. The value of h is a set value, determined according to the installation location of the image acquisition device. For details, please refer to [link to relevant documentation]. Figure 4 .

[0106] As an example, if the feature information includes the height of the obstacle, such as... Figure 5 As shown, determining the feature information of the obstacle based on the position information of the second image in the first image may include the following steps:

[0107] S51. Determine the total number of rows of pixels in the second image.

[0108] The total number of rows of pixels in the second image is its vertical resolution. For example, if the resolution of the second image is 600*200, then the total number of rows of pixels in the second image is 200. Figure 4 As shown, L2 represents the total number of rows of pixels in the second image.

[0109] S52. Determine the longitudinal proportion angle of the obstacle based on the total number of pixel rows in the second image, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device.

[0110] like Figure 4 As shown, ε2 represents the longitudinal proportion angle of the obstacle.

[0111] As an example, the vertical proportion angle can be calculated using the following formula:

[0112] ε2=β*(L2 / Row)

[0113] S53. Determine the ground projection distance between the upper edge of the obstacle and the image acquisition device based on the first included angle, the second included angle, the longitudinal proportion angle, and the distance between the image acquisition device and the ground.

[0114] See Figure 4 d2 represents the ground projection distance between the upper edge of the obstacle and the image acquisition device.

[0115] As an example, the ground projection distance between the upper edge of the obstacle and the image acquisition device can be calculated according to the following formula:

[0116] d2=h*tan(δ+ε+ε2).

[0117] S54. Determine the height of the obstacle based on the first included angle, the second included angle, the ground projection distance between the upper edge of the obstacle and the image acquisition device, and the ground projection distance between the obstacle and the image acquisition device.

[0118] As one example, the height of an obstacle can be calculated using the following formula:

[0119] h2=(d2-d1) / tan(δ+ε+ε2)

[0120] Where h2 represents the height of the obstacle.

[0121] As one possible implementation, if the feature information includes the width of the obstacle, see [link to relevant documentation]. Figure 6The step of determining the feature information of the obstacle based on the position information of the second image in the first image may include the following steps:

[0122] S61. Determine the total number of columns of pixels in the second image.

[0123] The total number of columns of pixels in the second image is its horizontal resolution. For example, if the resolution of the second image is 600*200, then the total number of columns of pixels in the second image is 600. Figure 4 As shown, L3 indicates the number of third pixels.

[0124] S62. Determine the lateral proportion angle of the obstacle based on the total number of pixel columns in the second image, the total number of pixel columns in the first image, and the visible horizontal angle of the image acquisition device.

[0125] See also Figure 4 θ represents the lateral proportion of the obstacle.

[0126] As an example, the lateral proportion angle of the obstacle can be calculated using the following formula:

[0127] θ = ρ * (L3 / Column)

[0128] Where θ represents the lateral proportion angle, ρ represents the horizontal viewing angle of the image acquisition device, L3 represents the total number of pixel columns in the second image, and Column represents the total number of pixel columns in the first image.

[0129] S63. Determine the width of the obstacle based on the lateral proportion angle and the ground projection distance between the obstacle and the image acquisition device.

[0130] As an example, the width of the obstacle can be calculated using the following formula:

[0131] W = 2 * d1 * tan(θ / 2)

[0132] Where W represents the width of the obstacle.

[0133] This invention provides an obstacle detection method. By acquiring a first image of a detection area, and when an obstacle exists in the detection area, a second image corresponding to the obstacle is extracted from the first image. The feature information of the obstacle is determined based on the position information of the second image in the first image. This solution can determine the obstacle's own feature information based on the image corresponding to the obstacle. Since the obstacle's feature information reflects its characteristics, obstacles can be distinguished based on the acquired feature information, thus solving the current technical problem of being unable to distinguish obstacles.

[0134] Figure 7 A flowchart of another obstacle detection method provided in an embodiment of the present invention is shown below. Figure 7 As shown, the method may include the following steps:

[0135] S71. Obtain the first image of the detection area.

[0136] S72. If it is determined that there is an obstacle in the detection area, extract the second image corresponding to the obstacle from the first image.

[0137] S73. Determine the feature information of the obstacle based on the position information of the second image in the first image.

[0138] Steps S71-S73 can be found in the descriptions of S11-S13, and will not be repeated here.

[0139] S74. Determine the type of obstacle based on the feature information.

[0140] As an alternative implementation, if the first image is a temperature distribution image and the feature information includes the width and height of the obstacle, the type of obstacle can be determined in the following way:

[0141] Based on the width and height of the obstacle, the aspect ratio of the obstacle is determined, whether the obstacle is moving is detected, and the surface temperature of the obstacle is determined. If the aspect ratio is less than a preset first threshold, the obstacle is moving, and the surface temperature is within a preset temperature range, then the obstacle is determined to be a human object; otherwise, the obstacle is determined to be a non-human object.

[0142] Both the first threshold and the preset temperature range are pre-set. Specifically, when setting the first threshold, the height and width of multiple people can be collected, and then the aspect ratio of each person can be calculated. The average aspect ratio of all people can then be calculated, and the final average aspect ratio is determined as the first threshold. For example, the first threshold could be 0.4. When setting the preset temperature range, the body temperature values ​​of multiple people can be collected, and then a suitable temperature range can be set based on these values. This temperature range is then used as the preset temperature range.

[0143] As one embodiment, the movement of the obstacle can be detected in the following manner:

[0144] Multiple first images of the detection area are continuously acquired, and the second image corresponding to the obstacle in each first image is extracted to obtain multiple second images. For each second image, its position in the corresponding first image is determined. If the positions of multiple second images in their corresponding first images are all the same, it is determined that the obstacle has not moved; otherwise, it is determined that the obstacle has moved.

[0145] The process of continuously acquiring multiple images of the detection area can be achieved by acquiring images of the detection area once every second preset time interval within a first preset time interval, while keeping the position of the image acquisition device unchanged. The first and second preset time intervals can be set according to actual needs, but the first preset time interval must be longer than the second preset time interval. For example, the first preset time interval can be 1 minute and the second preset time interval can be 10 seconds.

[0146] Because the position of the image acquisition device remains unchanged, the image acquisition area remains unchanged. If the position of the second image in the first acquired image changes, it indicates that the position of the obstacle has changed, and thus it can be determined that the obstacle has moved.

[0147] As one embodiment, the surface temperature of an obstacle can be determined in the following manner:

[0148] The first image is a temperature distribution image, so each pixel in the second image has a corresponding temperature value. The average temperature of all pixels in the second image is calculated, and the calculated average temperature is used as the surface temperature of the obstacle.

[0149] Furthermore, after determining that the obstacle is a non-human object, the type of obstacle can be further determined in the following ways:

[0150] Determine whether the shape of the second image is a preset shape. If the shape of the second image is determined to be a preset shape, then according to the preset correspondence between the aspect ratio of an object and the object type, determine the target type corresponding to the aspect ratio of the obstacle, and determine the target type as the type of the obstacle.

[0151] As an example, the preset shape can be a solid rectangle. When the second image is determined to be the preset shape, the obstacle can be initially determined to be a regular object such as a sofa or cabinet. Then, the specific type of obstacle can be further determined according to the aspect ratio of the obstacle. For example, if the aspect ratio belongs to the first interval, the obstacle is determined to be a sofa. If the aspect ratio belongs to the second region, the obstacle is determined to be a cabinet. The first interval can be [2, 4], and the second region can be [0.2, 1]. The above two intervals are just examples, and the specific ones can be set according to actual needs.

[0152] If the second image is not a preset shape, then the obstacle can be determined to be an irregularly shaped object, such as a coffee table or a chair.

[0153] As another alternative implementation, the type of obstacle can be determined in the following way:

[0154] The feature information is input into a pre-trained object classification model, which then outputs the type of obstacle.

[0155] The feature information may include the height and width of the obstacle.

[0156] As an example, an object classification model can be trained in the following manner:

[0157] The height and width of multiple known types of obstacles are collected. The height and width of the same obstacle are used as a sample data. A category label is added to each sample data. Then, using multiple labeled sample data, a classification model is trained using KNN (K-nearest neighbor clustering algorithm) or SVM (Support Vector Machine). The trained classification model is used as the trained object classification model.

[0158] This embodiment provides an obstacle detection method that classifies obstacles based on their characteristic information, and can distinguish obstacles according to their type.

[0159] When the obstacle detection method provided by this invention is applied to an air conditioner, the feature information of the obstacle may also include the distance between the obstacle and the image acquisition device.

[0160] As an optional implementation, the ground projection distance between the obstacle and the image acquisition device can be directly used as the distance between the obstacle and the image acquisition device.

[0161] As another optional implementation, if the first image is a temperature distribution map and the image acquisition device is located near the air outlet of the air conditioner, the distance between the obstacle and the image acquisition device can be calculated based on the surface temperature of the obstacle and the air outlet temperature of the air conditioner. The average temperature of the pixels in the second image can then be used as the surface temperature of the obstacle. Specifically, the distance between the obstacle and the image acquisition device can be calculated using the following formula:

[0162]

[0163] Where d0 represents the distance between the obstacle and the image acquisition device, T 出 This indicates the air outlet temperature of the air conditioner, T. 表 The surface temperature of the obstacle is represented by k, which is a constant and represents the temperature decay rate.

[0164] As another optional implementation, the distance between the obstacle and the image acquisition device can be calculated using the two methods described above, and then further calculated based on the following formula. The result of the calculation using the following formula can be used as the final distance between the obstacle and the image acquisition device:

[0165] d = K * d1 + (1 - K) * d0

[0166] Where K is a preset weight value based on actual needs, and d1 is the ground projection distance between the obstacle and the image acquisition device.

[0167] This method combines the results of the two calculations, resulting in a more accurate final distance value.

[0168] The above is a general description of the obstacle detection method provided in the embodiments of the present invention. The following description, in conjunction with the appendix, provides further details. Figure 8 The method for determining whether there are obstacles in the detection area when the first image is a temperature distribution image is described.

[0169] like Figure 8 The process of determining whether there are obstacles within the detection area includes the following steps:

[0170] S81. Divide the first image into a first sub-image and a second sub-image.

[0171] As an example, an image segmentation rule is preset, and the first image is divided according to the preset image segmentation rule to obtain a first sub-image and a second sub-image. The image segmentation rule can be set according to actual needs or experience. For example, the preset image segmentation rule can be: divide the first image acquired by the infrared array sensor at half of the vertical pixels (i.e., 2 / Row) to obtain an upper half image and a lower half image. The upper half image is used as the first sub-image, and the lower half image is used as the second sub-image.

[0172] S82. Determine the average temperature of all pixels in the first sub-image as the first temperature.

[0173] S83. Determine the average temperature of all pixels in the second sub-image as the second temperature.

[0174] S84. Determine the temperature difference between the first temperature and the second temperature.

[0175] S85. Determine the temperature variance corresponding to each pixel in the first image.

[0176] S86. Determine whether the temperature difference is less than the first temperature difference and whether the temperature variance is less than the first threshold. If the temperature difference is less than the first temperature difference and the temperature variance is less than the first threshold, then execute S811; otherwise, execute S88.

[0177] The first temperature difference and the first threshold are both values ​​set according to actual needs.

[0178] S87. Determine the maximum temperature value in the temperature distribution map.

[0179] S88. Determine the ratio of the maximum temperature value to the air outlet temperature of the air conditioner.

[0180] S89. Determine whether the ratio is less than a first preset ratio and whether the temperature difference is less than a second temperature difference. If the ratio is less than the first preset ratio and the temperature difference is less than the second temperature difference, then execute S811; otherwise, execute S810.

[0181] The first preset ratio and the second temperature difference are values ​​set according to actual needs, wherein the second temperature difference is greater than or equal to the first temperature difference.

[0182] S810. Determine that there is an obstacle in the detection area.

[0183] S811. Determine that there are no obstacles in the detection area.

[0184] The following section provides a unified explanation of S82-S811, using the application scenario of this solution in an air conditioner:

[0185] As an example, taking the detection area as the air supply range of the air conditioner during heating operation, when the air conditioner is heating, the air conditioner vents air through the lower air outlet. If there are no obstacles in front of the lower air outlet, that is, within the air supply range, and the air outlet path is not blocked, the hot air moves forward and then rises and diffuses to the entire room. The overall temperature of the room will be relatively uniform, and there will be no obvious temperature difference in the first image collected by the infrared array sensor. The first temperature corresponding to the first sub-image and the second temperature corresponding to the second sub-image will be relatively close, the temperature difference will not be greater than the first temperature difference, and the temperature variance of each pixel will not be greater than the first threshold.

[0186] If the temperature difference is greater than the first temperature difference value, or the temperature variance of each pixel is greater than the first threshold, it indicates that there may be an obstacle. To further ensure the accuracy of the final detection result, when the temperature difference is greater than the first temperature difference value, or the temperature variance of each pixel is greater than the first threshold, another method can be used for obstacle detection, i.e., the method in steps S87-S89 can be executed.

[0187] If there are obstacles blocking the air supply range during heating, the air outlet path will be blocked, causing the hot air blown from the lower air outlet to blow directly onto the obstacle. Furthermore, due to the obstruction of the obstacle, the hot air cannot diffuse smoothly to the entire room, resulting in the temperature near the obstacle being significantly higher than the temperature in other areas. Of course, the maximum temperature rise of the obstacle surface will not exceed the temperature of the air outlet. Therefore, the presence of an obstacle can be determined by the ratio of the maximum temperature value in the temperature distribution image to the air outlet temperature (or the air outlet temperature). If the ratio of the maximum temperature value to the air outlet temperature is greater than the first preset ratio, and the temperature difference between the first temperature and the second temperature is greater than the second temperature difference, then an obstacle is considered to be present.

[0188] By combining the two methods described above to detect obstacles, the accuracy of the detection results is ensured. Of course, the two methods can also be used individually. For example, obstacle detection can be performed through S81-S86. When it is determined through S86 that the temperature difference is not less than the first temperature difference or the temperature variance is not less than the first threshold, it is determined that there is an obstacle in the air supply range. Alternatively, obstacle detection can be performed through S81-S84 and S87-S89. When it is determined through S89 that the ratio of the maximum temperature value to the air outlet temperature of the air conditioner is greater than the first preset ratio, and the temperature difference between the first temperature and the second temperature is greater than the second temperature difference, it is determined that there is an obstacle in the air supply range.

[0189] This invention also provides embodiments of an obstacle detection device, which can be applied to air conditioners, such as... Figure 9 As shown, the device may include:

[0190] Detection module 901 is used to acquire the first image of the detection area;

[0191] Extraction module 902 is used to extract a second image corresponding to an obstacle from the first image when it is determined that there is an obstacle in the detection area;

[0192] The determination module 903 determines the feature information of the obstacle based on the position information of the second image in the first image.

[0193] As one possible implementation, the feature information includes the ground projection distance between the obstacle and the image acquisition device corresponding to the first image, and the determining module 903 is specifically used for:

[0194] Determine the number of first pixel rows that differ between the bottom edge of the second image and the bottom edge of the first image;

[0195] Based on the first pixel row number, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device corresponding to the first image, determine the first included angle between the lower edge of the obstacle and the lower edge of the frame of the image acquisition device;

[0196] Based on the included angle between the image acquisition device and the mounting surface and the visible pitch angle of the image acquisition device, determine the second included angle between the lower edge of the frame of the image acquisition device and the mounting surface;

[0197] Based on the first intersection angle, the second included angle, and the distance between the image acquisition device and the ground, the ground projection distance between the obstacle and the image acquisition device is determined.

[0198] As one possible implementation, based on the first angle, the second included angle, and the distance between the image acquisition device and the ground, the ground projection distance between the obstacle and the image acquisition device is determined according to the following formula:

[0199] d1 = h * tan(δ + ε)

[0200] δ=(π / 2–(β / 2+α))

[0201] ε=β*(L1 / Row)

[0202] Where d1 represents the ground projection distance between the obstacle and the image acquisition device, h represents the distance between the image acquisition device and the ground, δ represents the second included angle, β represents the visible pitch angle of the image acquisition device, α represents the included angle between the image acquisition device and the mounting surface, ε represents the first included angle, L1 represents the first pixel row number, and Row represents the total number of pixel rows of the first image.

[0203] As one possible implementation, the feature information also includes the height of the obstacle, and the determining module 903 is specifically used for:

[0204] Determine the total number of rows of pixels in the second image;

[0205] The longitudinal proportion angle of the obstacle is determined based on the total number of pixel rows in the second image, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device.

[0206] The ground projection distance between the upper edge of the obstacle and the image acquisition device is determined based on the first included angle, the second included angle, the longitudinal proportion angle, and the distance between the image acquisition device and the ground.

[0207] The height of the obstacle is determined based on the first included angle, the second included angle, the ground projection distance between the upper edge of the obstacle and the image acquisition device, and the ground projection distance between the obstacle and the image acquisition device.

[0208] As one possible implementation, the height of the obstacle is determined based on the first included angle, the second included angle, the ground projection distance between the upper edge of the obstacle and the image acquisition device, and the ground projection distance between the obstacle and the image acquisition device, including:

[0209] h2=(d2-d1) / tan(δ+ε+ε2)

[0210] ε2=β*(L2 / Row)

[0211] d² = h * tan(δ + ε + ε²)

[0212] Where h2 represents the height of the obstacle, d2 represents the ground projection distance between the upper edge of the obstacle and the image acquisition device, ε2 represents the longitudinal proportion angle, and L2 represents the total number of pixel rows in the second image.

[0213] As one possible implementation, the feature information also includes the width of the obstacle, and the determining module 903 is specifically used for:

[0214] Determine the total number of columns of pixels in the second image;

[0215] The lateral proportion angle of the obstacle is determined based on the total number of pixel columns in the second image, the total number of pixel columns in the first image, and the visible horizontal angle of the image acquisition device.

[0216] The width of the obstacle is determined based on the lateral proportion angle and the ground projection distance between the obstacle and the image acquisition device.

[0217] As one possible implementation, the width of the obstacle is determined according to the following formula, based on the lateral proportion angle and the ground projection distance between the obstacle and the image acquisition device:

[0218] W = 2 * d1 * tan(θ / 2)

[0219] θ = ρ * (L3 / Column)

[0220] Wherein, W represents the width of the obstacle, θ represents the lateral proportion angle, ρ represents the horizontal viewing angle of the image acquisition device, L3 represents the total number of pixel columns in the second image, and Column represents the total number of pixel columns in the first image.

[0221] As one possible implementation, the device further includes a classification module ( Figure 9 (not shown in the image), specifically used for:

[0222] The aspect ratio of the obstacle is determined based on its width and height.

[0223] Determine whether the obstacle has moved;

[0224] Determine the surface temperature of the obstacle;

[0225] If the aspect ratio is less than a preset first threshold, the obstacle moves, and the surface temperature is within a preset temperature range, then the obstacle is determined to be a human; otherwise, the obstacle is determined to be a non-human object.

[0226] As one possible implementation, the classification module is also used for:

[0227] If the obstacle is determined to be a non-human object, then determine whether the shape of the second image is a preset shape;

[0228] If the shape of the second image is determined to be a preset shape, then according to the preset correspondence between the aspect ratio of an object and the object type, the target type corresponding to the aspect ratio of the obstacle is determined, and the target type is determined to be the type of the obstacle.

[0229] As one possible implementation, the classification module is specifically used for:

[0230] The feature information is input into a pre-trained object classification model, which then outputs the type of obstacle.

[0231] In another embodiment of this application, an electronic device is also provided, such as Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002 and the memory 1003 communicate with each other through the communication bus 1004.

[0232] Memory 1003 is used to store computer programs;

[0233] When processor 1001 executes a program stored in memory 1003, it performs the following steps:

[0234] Acquire the first image of the detection area;

[0235] If an obstacle is detected within the detection area, a second image corresponding to the obstacle is extracted from the first image;

[0236] Based on the position information of the second image in the first image, the feature information of the obstacle is determined.

[0237] The communication bus 1004 mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1004 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0238] The communication interface 1002 is used for communication between the above-mentioned electronic device and other devices.

[0239] The memory 1003 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0240] The processor 1001 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0241] In another embodiment of this application, a storage medium is also provided, which stores one or more programs that can be executed by one or more processors to implement the steps of any of the obstacle detection methods described above.

[0242] In specific implementation, the embodiments of the present invention can be referred to the above embodiments, and have corresponding technical effects.

[0243] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0244] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An obstacle detection method, characterized in that, include: Acquire the first image of the detection area. If the first image of the detection area is a temperature distribution image, determine whether there is an obstacle in the detection area based on the temperature value corresponding to the pixel in the first image. If an obstacle is detected within the detection area, a second image corresponding to the obstacle is extracted from the first image; The feature information includes: the ground projection distance between the obstacle and the image acquisition device corresponding to the first image, and the height of the obstacle. Based on the position information of the second image in the first image, the feature information of the obstacle is determined, including: Determine the number of first pixel rows that differ between the bottom edge of the second image and the bottom edge of the first image; Based on the first pixel row number, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device corresponding to the first image, determine the first included angle between the lower edge of the obstacle and the lower edge of the frame of the image acquisition device; Based on the included angle between the image acquisition device and the mounting surface and the visible pitch angle of the image acquisition device, determine the second included angle between the lower edge of the frame of the image acquisition device and the mounting surface; Based on the first included angle, the second included angle, and the distance between the image acquisition device and the ground, the ground projection distance between the obstacle and the image acquisition device is determined. Determine the total number of rows of pixels in the second image; The longitudinal proportion angle of the obstacle is determined based on the total number of pixel rows in the second image, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device. The ground projection distance between the upper edge of the obstacle and the image acquisition device is determined based on the first included angle, the second included angle, the longitudinal proportion angle, and the distance between the image acquisition device and the ground. The height of the obstacle is determined based on the first included angle, the second included angle, the ground projection distance between the upper edge of the obstacle and the image acquisition device, and the ground projection distance between the obstacle and the image acquisition device, including: in, Indicates the height of the obstacle. This represents the ground projection distance between the upper edge of the obstacle and the image acquisition device. This indicates the vertical proportion angle. This represents the total number of rows of pixels in the second image; The feature information also includes the width of the obstacle, and determining the feature information of the obstacle based on the position information of the second image in the first image includes: Determine the total number of columns of pixels in the second image; The lateral proportion angle of the obstacle is determined based on the total number of pixel columns in the second image, the total number of pixel columns in the first image, and the visible horizontal angle of the image acquisition device. The width of the obstacle is determined based on the lateral proportion angle and the ground projection distance between the obstacle and the image acquisition device, including: in, Indicates the width of the obstacle. This indicates the horizontal proportion angle. This indicates the horizontal viewing angle of the image acquisition device. This represents the total number of columns of pixels in the second image. This represents the total number of columns of pixels in the first image.

2. The method according to claim 1, characterized in that, Based on the first included angle, the second included angle, and the distance between the image acquisition device and the ground, the ground projection distance between the obstacle and the image acquisition device is determined according to the following formula: Where d1 represents the ground projection distance between the obstacle and the image acquisition device, and h represents the distance between the image acquisition device and the ground. Indicates the second included angle. This indicates the visible pitch angle of the image acquisition device. This indicates the angle between the image acquisition device and the mounting surface. Indicates the first included angle. Represents the number of the first pixel row, This represents the total number of rows of pixels in the first image.

3. The method according to claim 1, characterized in that, The method further includes: The aspect ratio of the obstacle is determined based on its width and height. Determine whether the obstacle has moved; Determine the surface temperature of the obstacle; If the aspect ratio is less than a preset first threshold, the obstacle moves, and the surface temperature is within a preset temperature range, then the obstacle is determined to be a human; otherwise, the obstacle is determined to be a non-human object.

4. The method according to claim 3, characterized in that, The method further includes: If the obstacle is determined to be a non-human object, then determine whether the shape of the second image is a preset shape; If the shape of the second image is determined to be a preset shape, then according to the preset correspondence between the aspect ratio of an object and the object type, the target type corresponding to the aspect ratio of the obstacle is determined, and the target type is determined to be the type of the obstacle.

5. The method according to claim 1, characterized in that, The method further includes: The feature information is input into a pre-trained object classification model, which then outputs the type of obstacle.

6. An obstacle detection device, characterized in that, include: The acquisition module is used to acquire the first image of the detection area. If the first image of the detection area is a temperature distribution image, the presence of an obstacle in the detection area is determined based on the temperature value corresponding to the pixel in the first image. The extraction module is used to extract a second image corresponding to an obstacle from the first image when it is determined that there is an obstacle in the detection area; The feature information includes: the ground projection distance between the obstacle and the image acquisition device corresponding to the first image, and the height of the obstacle. The determination module is used to determine the feature information of the obstacle based on the position information of the second image in the first image, including: determining the number of first pixel rows that differ between the lower edge of the second image and the lower edge of the first image. Based on the first pixel row number, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device corresponding to the first image, determine the first included angle between the lower edge of the obstacle and the lower edge of the frame of the image acquisition device; Based on the included angle between the image acquisition device and the mounting surface and the visible pitch angle of the image acquisition device, determine the second included angle between the lower edge of the frame of the image acquisition device and the mounting surface; Based on the first included angle, the second included angle, and the distance between the image acquisition device and the ground, the ground projection distance between the obstacle and the image acquisition device is determined. Determine the total number of rows of pixels in the second image; The longitudinal proportion angle of the obstacle is determined based on the total number of pixel rows in the second image, the total number of pixel rows in the first image, and the visible pitch angle of the image acquisition device. The ground projection distance between the upper edge of the obstacle and the image acquisition device is determined based on the first included angle, the second included angle, the longitudinal proportion angle, and the distance between the image acquisition device and the ground. The height of the obstacle is determined based on the first included angle, the second included angle, the ground projection distance between the upper edge of the obstacle and the image acquisition device, and the ground projection distance between the obstacle and the image acquisition device, including: in, Indicates the height of the obstacle. This represents the ground projection distance between the upper edge of the obstacle and the image acquisition device. This indicates the vertical proportion angle. This represents the total number of rows of pixels in the second image; The feature information also includes the width of the obstacle, and determining the feature information of the obstacle based on the position information of the second image in the first image includes: Determine the total number of columns of pixels in the second image; The lateral proportion angle of the obstacle is determined based on the total number of pixel columns in the second image, the total number of pixel columns in the first image, and the visible horizontal angle of the image acquisition device. The width of the obstacle is determined based on the lateral proportion angle and the ground projection distance between the obstacle and the image acquisition device, including: in, Indicates the width of the obstacle. This indicates the horizontal proportion angle. This indicates the horizontal viewing angle of the image acquisition device. This represents the total number of columns of pixels in the second image. This represents the total number of columns of pixels in the first image.

7. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a data processing program stored in the memory to implement the obstacle detection method of any one of claims 1-5.

8. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the obstacle detection method according to any one of claims 1-5.

Images