Target positioning method and system based on stereoscopic depth data

Abstract

The invention provides a target positioning method and system based on three-dimensional depth data, and the method comprises the steps: determining a target space region which is a region where a target object is not allowed to enter or leave; according to the target space area, arranging at least one stereo depth sensor to collect real-time position data of the target object; according to the position relationship between the target object and the target space area, determining whether a prompt needs to be sent; if a prompt needs to be sent out, the image data of the target object is obtained through the optical image sensor, the prompt is given, the spatial orientation of the specific target is monitored through the three-dimensional depth data, and early warning or warning on the situation that the specific target breaks away from or enters some special spatial areas within the specific time is achieved; therefore, the purpose of equivalence with fence arrangement is achieved. Facilities or equipment for data acquisition, acousto-optic warning, scene recording and the like involved in the technology have relatively high mobility; the device can be repeatedly used; the positioning precision is high; and evidence obtaining and retention are convenient.

Description

Technical Field

[0001] This article belongs to the field of sensor positioning technology, and specifically relates to a target positioning method and system based on stereo depth data. Background Art

[0002] In some special spatial areas such as construction sites, military restricted areas, hazardous chemical production and storage, etc., it is necessary to locate external targets in order to limit the range of activities of specific targets in the area. At present, the more commonly used spatial area restriction control method is line drawing warning, that is, using marks or lines on the ground to demarcate specific activity areas to achieve control over these areas. For example, in some construction sites, production workshops, etc., lines are drawn to remind people not to enter or leave these areas. Figure 1 The figure shows a line drawing on the ground of a commonly used space restriction area. The area represented by the line drawing, that is, the area from the ground upwards is a restricted entry or exit area, which is used to remind relevant personnel not to leave or enter the area. This method of drawing lines for warning is simple and easy to implement. However, whether the relevant personnel comply with it depends entirely on their own consciousness. When an accident occurs, it is difficult to distinguish the relevant responsibilities. In order to achieve effective control, these areas need to be separated from other areas. The commonly used measure at present is to build fences, that is, to install and deploy fences in these special areas to mark the area and define the target activity range.

[0003] In the prior art, various simple fences or smart fences are used to locate specific targets and limit the target's range of movement in the area. Most products have the disadvantages of poor mobility, low positioning accuracy, non-reusability, high cost, and no target recognition. Therefore, a practical solution that can quickly and accurately locate the target object is urgently needed. Summary of the invention

[0004] In view of the above problems in the prior art, the purpose of this article is to provide a target positioning method and system based on stereo depth data, which can improve the accuracy of positioning target objects in a specific area.

[0005] In order to solve the above technical problems, the specific technical solutions of this article are as follows:

[0006] On the one hand, this article provides a target positioning method based on stereo depth data, the method comprising:

[0007] Determine a target space area, where the target space area is an area that the target object is not allowed to enter or leave;

[0008] Arrange at least one stereo depth sensor according to the target spatial area to collect real-time position data of the target object;

[0009] Determine whether a prompt needs to be issued according to the positional relationship between the target object and the target space region;

[0010] If a prompt needs to be issued, obtain the image data of the target object through the optical image sensor and give a prompt.

[0011] Further, when the number of stereo depth sensors is 1, the acquisition range of the stereo depth sensor at least includes the target space region; the method further includes:

[0012] Obtain the target position data (a ij , j, i)) collected by the stereo depth sensor, and the position range (X, Y, Z) of the target space region, where a ij represents the straight-line distance from the stereo depth sensor to the target object, and i, j are the azimuths of the image pixels in the matrix, X1 < X < X2, Y1 < Y < Y2, Z1 < Z < Z2;

[0013] When the target space region is an area where the target object is not allowed to enter, the target position data needs to at least not satisfy the following condition: X1 < a ij < X2, Y1 < j < Y2, Z1 < i < Z2, otherwise, give a prompt;

[0014] When the target space region is an area where the target object is not allowed to leave, the target position data needs to satisfy the following conditions: X1 < a ij < X2, Y1 < j < Y2, Z1 < i < Z2, otherwise, give a prompt.

[0015] Further, the stereo depth sensor is arranged at the boundary position of the target space region.

[0016] Further, when the number of stereo depth sensors is multiple, the multiple stereo depth sensors are arranged at the boundary position of the target space region, and the acquisition range of each stereo depth sensor at least includes the target space region and the acquisition blind area of at least one other stereo depth sensor.

[0017] Further, the method further includes:

[0018] Establish a transformation matrix based on the same three-dimensional coordinate system according to the three-dimensional coordinate system where each stereo depth sensor is located;

[0019] According to the target position data collected by each stereo depth sensor and its corresponding transformation matrix, obtain the transformed target position data of each stereo depth sensor in the same three-dimensional coordinate system;

[0020] Determine whether to issue a prompt according to the range and type of the target space region, in combination with the transformed target position data of each stereo depth sensor.

[0021] Further, the multiple stereo depth sensors at least include a first stereo depth sensor and a second stereo depth sensor. The first stereo depth sensor and the second stereo depth sensor are oppositely arranged at the boundary of the target space region, and their acquisition fields of view are opposite.

[0022] The method further includes:

[0023] Establish a transformation matrix for converting the acquisition position data of the second stereo depth sensor to the coordinate system where the first stereo depth sensor is located, expressed as:

[0024] where, is the transformation matrix for converting the coordinate points in the second stereo depth sensor to the first stereo depth sensor, and L is the distance between the first stereo depth sensor and the second stereo depth sensor;

[0025] Convert the data collected by the second stereo depth sensor to the coordinate system where the first stereo depth sensor is located, expressed as:

[0026] (a' ij , j', i') are the data collected by the second stereo depth sensor, and (x1, y1, z1) are the coordinates after conversion;

[0027] When the target space region is an area where the target object is not allowed to enter, then at least one condition in each of the following two groups of conditions is not satisfied: –W / 2 < a ij < W / 2, 0 < j < L, –H / 2 < i < H / 2; or, –W / 2 < x1 < W / 2, 0 < y1 < L –H / 2 < z1 < –H / 2, otherwise, issue a prompt, where a ij represents the straight-line distance from the stereo depth sensor to the target object, i and j are the azimuths of the image pixels in the matrix, and L, W, and H are the length, width, and height of the target space region respectively;

[0028] When the target space region is an area where the target object is not allowed to leave, then both of the following two groups of conditions need to be satisfied: –W / 2 < a ij < W / 2, 0 < j < L, –H / 2 < i < H / 2; and, –W / 2 < x1 < W / 2, 0 < y1 < L –H / 2 < z1 < –H / 2, otherwise, issue a prompt.

[0029] Further, the stereo depth sensor is at least one of the following: a binocular optical camera, an infrared or laser ranging mechanism.

[0030] Furthermore, the prompt includes sending the image data of the target object to the user terminal and reminding the user via voice, email or telephone.

[0031] On the other hand, this article also provides a target positioning system based on stereo depth data, the system comprising:

[0032] at least one stereo depth sensor, arranged at a boundary of the target spatial region;

[0033] at least one optical image sensor corresponding one-to-one to the stereo depth sensor;

[0034] The processor is configured as:

[0035] Determine a target space area, where the target space area is an area where a target object of the stereo depth sensor is not allowed to enter or leave;

[0036] Arrange at least one stereo depth sensor according to the target spatial area to collect real-time position data of the target object;

[0037] Determining whether a prompt needs to be issued according to the positional relationship between the target object and the target spatial area;

[0038] If a prompt is required, the image data of the target object is obtained through an optical image sensor, and a prompt is given;

[0039] An alarm is used to issue an alarm prompt according to the processing result of the processor.

[0040] On the other hand, this document also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described above when executing the computer program.

[0041] Adopting the above technical scheme, a target positioning method and system based on stereo depth data described in this article includes: determining a target space area, the target space area is an area that the target object is not allowed to enter or leave; according to the target space area, at least one stereo depth sensor is arranged to collect the real-time position data of the target object; according to the positional relationship between the target object and the target space area, it is determined whether a prompt needs to be issued; if a prompt needs to be issued, the image data of the target object is obtained through an optical image sensor, and a prompt is issued. This article proposes to use stereo depth data to monitor the spatial orientation of a specific target, to achieve an early warning or warning of its departure or entry into certain special space areas within a specific time, thereby achieving the equivalent purpose of setting a fence. The data collection, sound and light warning, scene recording and other facilities or equipment involved in this technology have the following advantages: 1. High mobility; 2. Reusable; 3. High positioning accuracy; 4. Convenient evidence retention, etc.

[0042] In order to make the above and other purposes, features and advantages of this article more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of this article or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this article. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 Shows a commonly used ground line drawing of a space restriction area in the prior art;

[0045] Figure 2 A schematic diagram of the steps of a target positioning method based on stereo depth data provided in an embodiment of this invention is shown;

[0046] FIG3 shows a schematic diagram of ground warning line data collection in one embodiment of the present invention;

[0047] FIG4 is a schematic diagram showing a result of collecting ground warning line overhead data in one embodiment of the present invention;

[0048] Figure 5 A schematic diagram of collecting target stereo depth data in an embodiment of the present invention is shown;

[0049] FIG6 shows a schematic diagram of a stereo depth sensor acquisition in an embodiment of the present invention;

[0050] Figure 7 A flowchart of monitoring a single device prohibited from entering a space in an embodiment of the present invention is shown;

[0051] Figure 8 A flowchart of a single device being prohibited from leaving a space monitoring in one embodiment of this invention is shown;

[0052] Fig. 9 A schematic diagram of the placement of a binocular stereo depth camera in an embodiment of this invention is shown;

[0053] Fig.10 A flowchart of monitoring a space where two devices are prohibited from entering is shown in one embodiment of this document;

[0054] Fig.11 A flowchart of monitoring the prohibition of two devices from leaving the space in one embodiment of the present invention is shown;

[0055] Fig.12 A schematic diagram of the framework of a computer device provided in the embodiments of this document is shown. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this article to clearly and completely describe the technical solutions in the embodiments of this article. Obviously, the described embodiments are only part of the embodiments of this article, not all of the embodiments. Based on the embodiments of this article, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this article.

[0057] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0058] First, let’s explain the technical terms in the article:

[0059] Spatial relative positioning technology is a method of determining the relative position of an object in space, which is usually based on the relationship between the object and other objects or reference objects. This technology does not directly rely on an absolute coordinate system, but determines the position of an object by measuring and calculating the position of an object relative to other objects with known positions. Spatial relative positioning technology relies on the measurement and calculation of the relative position relationship between objects. These measurements may include distance, angle or other spatial parameters.

[0060] The implementation path is as follows:

[0061] 1. Selection of reference objects: First, you need to select one or more reference objects with known locations. These reference objects can be other objects or facilities that have been accurately located in space.

[0062] 2. Measure relative position: Use appropriate measuring equipment or technology, such as laser rangefinder, radar, etc., to measure the relative distance, angle or other relevant parameters between the target object and the reference object.

[0063] 3. Orientation data processing: Process and analyze the measured data, and determine the position of the target object relative to the reference object through geometric calculations or other algorithms.

[0064] Spatial relative positioning technology is widely used in various fields, including geographic information systems, remote sensing, global positioning systems, etc., and can accurately describe the position relationship of objects in three-dimensional space. Things to note when using this technology include:

[0065] 1. The accuracy of measurement directly affects the accuracy of positioning. Therefore, when selecting measurement equipment and methods, its accuracy and reliability need to be considered.

[0066] 2. The selection of reference objects is also very important. It is necessary to ensure that the position of the reference object is known and stable to reduce positioning errors.

[0067] 3. In complex environments, more reference objects and measurement parameters may need to be considered to improve the accuracy and robustness of positioning.

[0068] Spatial relative positioning technology is a method of positioning based on the relative relationship between objects. Its principle is simple but widely used. By selecting appropriate reference objects and measuring equipment, combined with appropriate data processing and analysis methods, high-precision spatial positioning can be achieved.

[0069] In the prior art, in some special spatial areas such as construction sites, military restricted areas, hazardous chemical production and storage, etc., it is necessary to locate external targets in order to limit the range of activities of specific targets in the area. The commonly used measure at present is to build fences, that is, to install and deploy fences in these special areas to mark the area and define the range of target activities. The following are several common devices, measures or methods that can achieve relative positioning of target space, and their respective advantages and disadvantages are analyzed as follows:

[0070] 1. Simple fence

[0071] Simple fences have a series of significant advantages. First, they are relatively low in cost, providing users with good cost-effectiveness. Secondly, they are easy to move, install and dismantle. This easy and quick deployment feature makes them very suitable for short-term projects. Simple fences, usually made of metal, are strong enough to effectively block unauthorized entry. However, simple fences also have some disadvantages. Due to their simple structure, their security is relatively low and they are not as strong as permanent fences. In addition, in order to ensure their stability, regular maintenance may be required to deal with possible loosening or damage.

[0072] 2. Electronic fence

[0073] As a security measure, although electronic fences are highly secure, can effectively prevent intruders, are suitable for the protection of large areas, and can be linked with security monitoring systems, their costs are relatively high, especially the initial installation and maintenance costs are relatively expensive. In addition, electronic fences are less mobile and usually require fixed installation. Once installed, they are often permanent, although the fence itself can be reused. However, this type of fence also has some obvious disadvantages, such as the installation process requires expertise, is costly, and its protective function may fail in the event of a power outage.

[0074] 3. Chain link fence

[0075] Chain link fences are an affordable option at a moderate cost. Although they are less mobile and are usually designed for permanent installation, they can be disassembled and reinstalled in different locations, which gives them a certain degree of flexibility. The advantage of this fence is that it is strong and durable, making it suitable for long-term use. At the same time, it provides safe and reliable protection because it is difficult to climb. In addition, the maintenance cost of chain link fences is relatively low, saving users subsequent expenses. However, it also has some disadvantages. First, the installation process can be complicated and often requires professional operation. Second, due to its inflexible nature, it is not suitable for temporary use or scenarios that require frequent changes in location.

[0076] 4. PVC Fence

[0077] PVC fencing is popular for its low cost, high mobility (depending on the design), and reusable properties. However, compared to metal fencing, PVC fencing generally has a shorter service life and durability. Despite this, PVC fencing has significant advantages, including being less prone to rust, low maintenance costs, customizable appearance and a variety of colors, and being lightweight and easy to install and move. However, it also has some limitations, such as being more suitable for areas with lower security requirements, its strength and durability are not as good as metal fencing, and long-term exposure to UV rays may cause fading.

[0078] 5. Hidden fence (such as living wall)

[0079] Concealed fencing, such as living walls, is a moderately expensive option, but it is less mobile and is usually a one-time construction, not reusable. However, it also has significant advantages: it is not only beautiful and environmentally friendly, it can naturally blend into the surrounding environment, provide privacy protection, and effectively reduce noise disturbance. However, this type of fence also has some obvious disadvantages. First, the maintenance cost is high, and a lot of manpower is required for regular pruning. Secondly, its establishment cost is relatively high, and the growth cycle is long, so it may take a long time to achieve the desired effect. Finally, due to the material and structural characteristics of this type of fence, its security is relatively low.

[0080] 6. Intelligent fence system

[0081] The smart fence system is a comprehensive security system that integrates sensors, cameras and other monitoring equipment. Although it is expensive, it provides the highest level of security. The system is usually fixed and therefore has poor mobility. However, it is worth noting that although the fixed facilities are not movable, the monitoring equipment has the flexibility to be reused in other places. The advantage of the smart fence system is that it can be integrated with existing security systems to achieve automated monitoring and alarms, thereby greatly improving the efficiency and accuracy of security monitoring. However, its disadvantages cannot be ignored, mainly including relatively high initial investment and maintenance costs, and the need for professional teams for management and maintenance.

[0082] These fences achieve the purpose of locating specific targets and limit the range of the target's activities in the area. Most products have the disadvantages of poor mobility, low positioning accuracy, non-reusability, high cost of use, and no target recognition.

[0083] In order to solve the above problems, the embodiments of this document provide a target positioning method based on stereo depth data, which can improve the accuracy of positioning a target object in a specific area. Figure 2 This is a schematic diagram of the steps of a target positioning method based on stereo depth data provided in the embodiments of this article. This specification provides the method operation steps described in the embodiments or flowcharts, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order of the methods shown in the embodiments or the drawings or in parallel. Specifically, Figure 2 As shown, the method may include:

[0084] S201: Determine a target space area, where the target space area is an area that the target object is not allowed to enter or leave;

[0085] S202: Arranging at least one stereo depth sensor according to the target spatial area to collect real-time position data of the target object;

[0086] S203: determining whether a prompt needs to be issued according to the positional relationship between the target object and the target spatial area;

[0087] S204: If a prompt is required, image data of the target object is obtained through an optical image sensor, and a prompt is issued.

[0088] It can be understood that the target space area in the embodiments of this specification is equivalent to a fence, and the function of the fence is to ensure that the monitored target does not leave or enter the space defined by the fence. Therefore, the purpose of setting a fence can be achieved by monitoring the relative position of a specific target and a special space area through spatial relative positioning technology, and restricting the specific target from leaving or entering the special space area. This article proposes the use of stereoscopic depth data to monitor the spatial orientation of a specific target, to achieve an early warning or warning of its departure from or entry into certain special space areas within a specific time, thereby achieving a purpose equivalent to setting a fence. The facilities or equipment involved in this technology, such as data collection, sound and light warnings, and scene recording, have the following advantages: 1. High mobility; 2. Reusable; 3. High positioning accuracy; 4. Convenient evidence collection and retention.

[0089] Exemplarily, the present invention mainly uses two types of data: one is spatial stereo depth data, which is generally collected by a stereo depth sensor using a binocular optical camera, infrared or laser ranging mechanism, and the data is the vertical distance from the target and background to the camera plane (point to plane to distance); the other is regional optical image information, which is generally collected by an optical image recording device using a CCD or CMOS sensor, and the data is the optical data of the light reflected or emitted by the target and background projected onto the optical sensor. Taking the collection of ground warning lines as an example, the process of collecting data by the dual-mode data collection device is as follows: Figure 3a , 3b and 3c, which are schematic diagrams of different views of ground warning line data collection; the collection results, i.e., ground warning line images and distance data matrix, are shown in FIG4, which are the results of ground warning line bird's-eye view data collection. Figure 4a Draw a line diagram for the ground. Figure 4b For optical data and Figure 4c is the degree data, where Figure 4b The matrix data shown in the view is the data collected by the optical sensor, and the pixel value a ij(i.e., x or y in the matrix) represents the light parameters (such as color, brightness, etc.) of the light rays incident on or reflected from a certain point of the target into the optical sensor, and i and j are the coordinates of the image pixels in the matrix. Figure 4c The matrix data shown in the view is the data collected by the depth sensor, and the value a of each element ij represents the straight-line distance from the camera to the target, and i and j are the coordinates of the image pixels in the matrix.

[0090] Furthermore, when the number of stereo depth sensors is 1, the acquisition range of the stereo depth sensor at least includes the target space region; the method further includes:

[0091] Obtaining the target position data (a ij , j, i)) collected by the stereo depth sensor, and the position range (X, Y, Z) of the target space region, where a ij represents the straight-line distance from the stereo depth sensor to the target object, i and j are the azimuths of the image pixels in the matrix, X1 < X < X2, Y1 < Y < Y2, Z1 < Z < Z2;

[0092] When the target space region is an area where the target object is not allowed to enter, the target position data needs to at least not satisfy the following condition: X1 < a ij < X2, Y1 < j < Y2, Z1 < i < Z2, otherwise, give a prompt;

[0093] When the target space region is an area where the target object is not allowed to leave, the target position data needs to satisfy the following conditions: X1 < a ij < X2, Y1 < j < Y2, Z1 < i < Z2, otherwise, give a prompt.

[0094] It can be understood that in most cases, the stereo depth sensor collects data at a frontal view angle, and the stereo depth sensor is set at the boundary position of the target space region. Taking the position of the stereo depth sensor as the origin O, the direction parallel to the ground in the front right direction of the sensor as the X-axis, the direction passing through the origin and parallel to the ground and perpendicular to the X-axis as the Y-axis, and the direction passing through the origin and perpendicular to the XOY plane upward as the Z-axis, as Figure 5 shown, it is a schematic diagram of the frontal view acquisition of the target stereo depth data.

[0095] As shown in Figure 6, the matrix data is a schematic diagram of the acquisition of the stereo depth sensor (binocular optical camera, infrared or laser ranging mechanism), where Figure 6a is a schematic diagram of the acquisition image of the target object. Figure 6b is the data of the target (i.e., a in the matrix) and the background (i.e., b in the matrix) acquired at a frontal view, and the pixel value a ij(i.e., a in the matrix) represents the straight-line distance from the camera to the target, and i and j are the positions of the image pixels in the matrix. Therefore, the fence in front of the stereo depth sensor delimits the area space, i.e. Figure 5 The double-line internal space S in the image can use coordinates (x, y, z). Assuming the target is in the double-line internal space, that is, the area enclosed by the fence, the camera faces the collected target pixel data (a ij , j, i)) should satisfy (in formula (1), X1, X2, Y1, Y2, Z1, Z2 are special spatial regions, i.e., the boundaries of the target spatial region):

[0096] X1 ij <X2、Y1<j<Y2、Z1<i<Z2 (1)

[0097] If only a single device is used for target monitoring, the stereo depth sensor can only measure the depth data of the target object facing the sensor, and cannot measure the depth data of the target object facing away from the sensor.

[0098] If the special space area (fenced area) is a prohibited entry space, the output of the stereo depth sensor is detected at regular intervals, that is, whether the pixel values ​​of the stereo depth data of the target facing the sensor meet formula (1). If all of them meet, it can be determined that the target has entered the prohibited entry space, and the target image is recorded by the optical sensor and an alarm is issued. The execution process is as follows: Figure 7 shown.

[0099] If the special space area (fenced area) is a prohibited space, the output of the stereo depth sensor is also detected, that is, whether the pixel values ​​of the stereo depth data of the target facing the sensor meet formula (1). If any one of the items is not satisfied, it means that the target has left the prohibited space, and the optical sensor will record the target image and alarm. The execution process is as follows: Figure 8 shown.

[0100] In some embodiments of the present specification, when there are multiple stereo depth sensors, the multiple stereo depth sensors are arranged at the boundary positions of the target space area, and the acquisition range of each stereo depth sensor at least includes the target space area and the acquisition blind area of ​​at least one remaining stereo depth sensor.

[0101] It can be understood that when using a single device for monitoring, the target can be quickly located, that is, the target that has left or entered the restricted space can be found in a relatively short time. If the target is large and its posture is constantly changing, it is impossible to determine whether the side of the monitored target facing away from the camera has left or entered the restricted space of the fence. Figure 5 ​As shown. When there are a large number of monitoring targets, the above situation will be more serious. By configuring the positions of the stereo depth sensors, the acquisition blind area of ​​each stereo depth sensor can be covered by the acquisition range of at least one other stereo depth sensor, thereby ensuring that a wider range can be collected.

[0102] Furthermore, the method further comprises:

[0103] According to the three-dimensional coordinate system where each stereo depth sensor is located, a transformation matrix based on the same three-dimensional coordinate system is established;

[0104] Obtain transformed target position data of each stereo depth sensor in the same three-dimensional coordinate system according to the target position data collected by each stereo depth sensor and its corresponding transformation matrix;

[0105] According to the range and type of the target space area, combined with the transformed target position data of each stereo depth sensor, it is determined whether to issue a prompt.

[0106] Further, the plurality of stereo depth sensors include at least a first stereo depth sensor and a second stereo depth sensor, the first stereo depth sensor and the second stereo depth sensor are relatively arranged at a boundary of the target space area, and have relative collection fields of view;

[0107] The method further comprises:

[0108] A transformation matrix is ​​established to convert the collected position data of the second stereo depth sensor to the coordinate system where the first stereo depth sensor is located, which is expressed as:

[0109] in, is a transformation matrix for converting coordinate points in the second stereo depth sensor to the first stereo depth sensor, and L is a distance between the first stereo depth sensor and the second stereo depth sensor;

[0110] The data collected by the second stereo depth sensor is converted into the coordinate system where the first stereo depth sensor is located, which is expressed as:

[0111] (a' ij , j', i') are the data collected by the second stereo depth sensor, and (x1, y1, z1) are the coordinates after conversion;

[0112] When the target space area is an area that the target object is not allowed to enter, at least one condition in each of the following two groups of conditions is not satisfied: –W / 2 ij ​<W / 2, 0 < j < L, –H / 2 < i < H / 2; or, –W / 2 < x1 < W / 2, 0 < y1 < L – H / 2 < z1 < –H / 2, otherwise, issue a prompt, where a ij represents the straight-line distance from the stereo depth sensor to the target object, i and j are the azimuths of the image pixels in the matrix, and L, W, and H are the length, width, and height of the target space region respectively;

[0113] When the target space region is an area where the target object is not allowed to break away, then the following two sets of conditions must be met: –W / 2 < a ij <W / 2, 0 < j < L, –H / 2 < i < H / 2; and, –W / 2 < x1 < W / 2, 0 < y1 < L – H / 2 < z1 < –H / 2, otherwise, issue a prompt.

[0114] It can be understood that to solve the defect of spatial relative positioning using a single binocular stereo depth camera, this solution uses two binocular stereo depth cameras for target spatial relative positioning. Let S be a special space region (such as Fig. 9 the internal space within the double wireframe in front of the stereo depth camera A, that is, the target space region), with its length, width, and height being L, W, and H respectively. The stereo depth cameras A and B are set at the midpoints of the two sides of S, as Figure 8 shown. Let the position of camera A be the origin O, and establish a three-dimensional O-XYZ coordinate system as Figure 8 shown. Then the points inside S can be represented by the coordinates (x, y, z) in the O-XYZ coordinate system. Suppose the target is in the internal space within the double wires, that is, within the space enclosed by the fence. Then the target pixel data (a ij , j, i)) collected by camera A facing forward should satisfy:

[0115] –W / 2 < a ij <W / 2, 0 < j < L, –H / 2 < i < H / 2 (2)

[0116] It can be determined that the part of the target facing stereo depth camera A will not break away from or enter the space defined by the fence, that is, the special space region. Camera B is set directly opposite camera A, with their viewfinders parallel and the lens axes coinciding. Let the position of camera B be the origin O', and establish a three-dimensional O'-X'Y'Z' coordinate system as Fig. 9 shown. According to the geometric relationship, we can obtain X = –X' + L, Y = –Y', Z = –Z', that is the transformation matrix for converting the coordinate points in the O'-X'Y'Z' coordinate system to the O-XYZ coordinate system, expressed as:

[0117]

[0118] Then the points inside S can use the coordinates (x', y', z') in the O'-X'Y'Z' coordinate system. Let the camera B look directly at the target pixel data collected in the O'-X'Y'Z' coordinate system in the form of an array (a' ij , j', i'). Convert it to the O-XYZ coordinate system and we get:

[0119]

[0120] Therefore, camera B faces the collected target pixel data (a' ij , j', i'), in the O-XYZ coordinate system it is (x1, y1, z1) satisfying:

[0121] –W / 2 <x1<W / 2, 0<y1<L,–H / 2<z1<–H / 2 (3)

[0122] It can be determined that the part of the target facing the stereo depth camera B will not leave or enter the space defined by the fence, that is, the special space area. If the special space area is a prohibited entry space, the output of the stereo depth sensor is detected at regular intervals, that is, whether the pixel values ​​of the target stereo depth data meet formulas 2 and 3. As long as one of them is satisfied, it means that a target has entered the prohibited entry space, and the target image is recorded by the optical sensor and an alarm is sounded. The execution process is as follows: Fig.10 shown.

[0123] If the special space area is a prohibited exit space, the output of the stereo depth sensor is also detected, that is, whether each pixel value of the target stereo depth data satisfies formulas 2 and 3. As long as one of the items is not satisfied, it means that the target has left the prohibited exit space, and the target image is recorded by the optical sensor and an alarm is issued. The execution process is as follows: Fig.11 shown.

[0124] Furthermore, the prompt includes sending the image data of the target object to the user terminal and reminding the user via voice, email or telephone.

[0125] This article describes a target positioning method and system based on stereo depth data, the method includes: determining a target space area, the target space area is an area that the target object is not allowed to enter or leave; according to the target space area, at least one stereo depth sensor is arranged to collect real-time position data of the target object; according to the positional relationship between the target object and the target space area, it is determined whether a prompt needs to be issued; if a prompt needs to be issued, the image data of the target object is obtained through an optical image sensor, and a prompt is issued. This article proposes to use stereo depth data to monitor the spatial orientation of a specific target, to achieve an early warning or warning of its departure or entry into certain special space areas within a specific time, thereby achieving the equivalent purpose of setting up a fence. The data collection, sound and light warning, scene recording and other facilities or equipment involved in this technology have the following advantages: 1. High mobility; 2. Reusable; 3. High positioning accuracy; 4. Convenient evidence retention, etc.

[0126] On the basis of the above-mentioned method, the embodiment of this specification further provides a target positioning system based on stereo depth data, the system comprising:

[0127] at least one stereo depth sensor, arranged at a boundary of the target spatial region;

[0128] at least one optical image sensor corresponding one-to-one to the stereo depth sensor;

[0129] The processor is configured as:

[0130] Determine a target space area, where the target space area is an area where a target object of the stereo depth sensor is not allowed to enter or leave;

[0131] Arrange at least one stereo depth sensor according to the target spatial area to collect real-time position data of the target object;

[0132] Determining whether a prompt needs to be issued according to the positional relationship between the target object and the target spatial area;

[0133] If a prompt is required, the image data of the target object is obtained through an optical image sensor, and a prompt is given;

[0134] An alarm is used to issue an alarm prompt according to the processing result of the processor.

[0135] The friendship effect obtained by the above system is consistent with the friendship effect obtained by the above method, and will not be described in detail in the embodiments of this specification.

[0136] This embodiment provides a computer device, whose internal structure diagram can be shown as follows: Fig.12As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection.

[0137] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0138] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0139] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0140] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0142] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0143] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0145] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0146] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0147] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for general technicians in this field, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on this article.

Claims

1. A target positioning method based on stereo depth data, characterized in that: The method comprises: Determine a target space area, where the target space area is an area that the target object is not allowed to enter or leave; Arrange at least one stereo depth sensor according to the target spatial area to collect real-time position data of the target object; Determining whether a prompt needs to be issued according to the positional relationship between the target object and the target spatial area; If a prompt is required, the image data of the target object is obtained through an optical image sensor and a prompt is given.

2. The method according to claim 1, characterized in that When the number of the stereo depth sensor is one, the acquisition range of the stereo depth sensor at least includes the target space area; the method further includes: Obtain target position data collected by the stereo depth sensor (a ij , j, i)), and the position range (X, Y, Z) of the target space area, where a ij represents the straight-line distance from the stereo depth sensor to the target object, i and j are the positions of the image pixels in the matrix, X1 <X<X2、Y1<Y<Y2、Z1<Z<Z2; When the target space area is an area where the target object is not allowed to enter, the target position data must at least not meet at least one of the following conditions: X1 < a ij <X2, Y1 < j < Y2, Z1 < i < Z2, otherwise, give a prompt; When the target spatial region is an area where the target object is not allowed to break away, the target position data shall meet the following conditions: X1 < a ij <X2, Y1 < j < Y2, Z1 < i < Z2, otherwise, a prompt shall be issued.

3. The method according to claim 2, characterized in that The stereo depth sensor is arranged at a boundary position of the target space area.

4. The method according to claim 1, characterized in that When there are multiple stereo depth sensors, the multiple stereo depth sensors are arranged at the boundary positions of the target space area, and the acquisition range of each stereo depth sensor at least includes the target space area and the acquisition blind area of ​​at least one remaining stereo depth sensor.

5. The method according to claim 4, characterized in that The method further comprises: According to the three-dimensional coordinate system where each stereo depth sensor is located, a transformation matrix based on the same three-dimensional coordinate system is established; Obtain transformed target position data of each stereo depth sensor in the same three-dimensional coordinate system according to the target position data collected by each stereo depth sensor and its corresponding transformation matrix; According to the range and type of the target space area, combined with the transformed target position data of each stereo depth sensor, it is determined whether to issue a prompt.

6. The method according to claim 5, characterized in that The plurality of stereo depth sensors include at least a first stereo depth sensor and a second stereo depth sensor, the first stereo depth sensor and the second stereo depth sensor are relatively arranged at a boundary of the target space area, and have relative collection fields of view; The method further comprises: A transformation matrix is ​​established to convert the collected position data of the second stereo depth sensor to the coordinate system where the first stereo depth sensor is located, which is expressed as: in, is a transformation matrix for converting coordinate points in the second stereo depth sensor to the first stereo depth sensor, and L is a distance between the first stereo depth sensor and the second stereo depth sensor; The data collected by the second stereo depth sensor is converted into the coordinate system where the first stereo depth sensor is located, which is expressed as: (a' ij , j', i') are the data collected by the second stereo depth sensor, and (x1, y1, z1) are the coordinates after conversion; When the target space region is an area where the target object is not allowed to enter, at least one condition in each of the following two groups of conditions is not satisfied: –W / 2 < a ij < W / 2, 0 < j < L, –H / 2 < i < H / 2; or, –W / 2 < x1 < W / 2, 0 < y1 < L –H / 2 < z1 < –H / 2, otherwise, a prompt is issued, where a ij represents the straight-line distance from the stereo depth sensor to the target object, i and j are the azimuths of the image pixels in the matrix, and L, W, and H are the length, width, and height of the target space region respectively; When the target spatial region is an area where the target object is not allowed to break away, the following two sets of conditions need to be satisfied: –W / 2 < a ij <W / 2, 0 < j < L, –H / 2 < i < H / 2; and, –W / 2 < x1 < W / 2, 0 < y1 < L – H / 2 < z1 < –H / 2, otherwise, a prompt is issued.

7. The method according to claim 1, characterized in that The stereo depth sensor is at least one of the following: a binocular optical camera, an infrared or laser ranging mechanism.

8. The method according to claim 1, characterized in that The prompt includes sending the image data of the target object to the user terminal and reminding through voice, email or telephone.

9. A target positioning system based on stereo depth data, characterized in that: The system comprises: at least one stereo depth sensor, arranged at a boundary of the target spatial region; at least one optical image sensor corresponding one-to-one to the stereo depth sensor; The processor is configured as: Determine a target space area, where the target space area is an area where a target object of the stereo depth sensor is not allowed to enter or leave; Arrange at least one stereo depth sensor according to the target spatial area to collect real-time position data of the target object; Determining whether a prompt needs to be issued according to a positional relationship between the target object and the target spatial area; If a prompt is required, the image data of the target object is obtained through an optical image sensor, and a prompt is given; An alarm is used to issue an alarm prompt according to the processing result of the processor.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images