Three-dimensional attribute detection method and device and storage medium

Abstract

The invention discloses a three-dimensional attribute detection method and device and a storage medium. The method comprises the steps that a sensor located at the top of a lift car is called in an elevator to detect and sense image data downwards; sensing that a pixel point in the image data has a depth value; performing self-correction on the depth values of the pixel points in the perception image data; if the self-correction is completed, detecting image data of a region where the target object is located in the sensing image data according to the lift car; and under the condition of filtering out the background of the lift car, carrying out statistics on the three-dimensional attribute of the target object in the regional image data. According to the embodiment, the data size is greatly reduced through operations such as target detection and background filtering, so that the calculation amount is effectively reduced, the method is suitable for hardware with low elevator calculation power, and the calculation efficiency is improved. Moreover, interference is gradually eliminated through operations such as target detection and background filtering, the target object is gradually converged, the three-dimensional attributes are counted for the target object, and the accuracy of the three-dimensional attributes can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of elevators, and in particular to a three-dimensional property detection method, device and storage medium. Background Art

[0002] In scenarios such as counting user density in an elevator car, three-dimensional data is collected from the car, the three-dimensional data is projected and transformed as a whole, restored to the world coordinate system, and the three-dimensional properties of the target object are calculated based on the world coordinate system.

[0003] However, this method requires a lot of computation, and elevators mostly use hardware with lower computing power (such as embedded devices), which results in low efficiency in calculations.

[0004] At the same time, there are many situations in which the relationship between different target objects (such as users) in the elevator car is close to each other, or one user's head is higher and partially covers the other user's head in the image, etc., which leads to deviations in the three-dimensional attributes. Summary of the invention

[0005] In view of this, the present invention provides a three-dimensional property detection method, device and storage medium, so as to improve the efficiency and accuracy of three-dimensional property detection in an elevator.

[0006] A first aspect of the present invention provides a method for detecting three-dimensional attributes, comprising:

[0007] In the elevator, a sensor located at the top of the car is called to detect downwardly sensed image data; the pixel points in the sensed image data have depth values;

[0008] performing self-correction on the depth value of the pixel in the perceived image data;

[0009] If the self-calibration is completed, then detecting the image data of the area where the target object is located in the sensed image data according to the car;

[0010] Under the condition of filtering out the car background, the three-dimensional attributes of the target object are counted in the regional image data.

[0011] A second aspect of the present invention provides a three-dimensional property detection device, comprising:

[0012] The car detection module is used to call the sensor located on the top of the car in the elevator to detect downward the perceived image data; the pixel points in the perceived image data have depth values;

[0013] A depth correction module, configured to self-correct the depth value of the pixel in the perceived image data;

[0014] A target detection module, for detecting, if self-calibration is completed, image data of a region where a target object is located in the sensed image data according to the car;

[0015] The attribute statistics module is used to count the three-dimensional attributes of the target object in the regional image data under the condition of filtering out the car background.

[0016] A third aspect of the present invention provides an electronic device, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the three-dimensional attribute detection method as described in the first aspect above.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the three-dimensional attribute detection method as described in the first aspect above.

[0021] A fifth aspect of the present invention provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the method for detecting three-dimensional attributes as described in the first aspect above is implemented.

[0022] In this embodiment, a sensor located at the top of the elevator car is called to detect downward the perceived image data; the pixels in the perceived image data have depth values; the depth values ​​of the pixels in the perceived image data are self-corrected; if the self-correction is completed, the regional image data where the target object is located is detected based on the car in the perceived image data; under the condition of filtering out the car background, the three-dimensional attributes of the target object are counted in the regional image data. This embodiment greatly reduces the amount of data through operations such as target detection and background filtering, thereby effectively reducing the amount of calculation, and is suitable for hardware with low computing power in elevators to improve its calculation efficiency. In addition, this embodiment gradually eliminates interference and gradually converges to the target object through operations such as target detection and background filtering, and counts the three-dimensional attributes of the target object, which can improve the accuracy of the three-dimensional attributes.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a flow chart of a three-dimensional attribute detection method provided in Embodiment 1 of the present invention.

[0026] Figure 2 This is an example diagram of perceptual image data provided by Embodiment 1 of the present invention.

[0027] Figure 3 This is an example diagram of detection area image data provided in Embodiment 1 of the present invention.

[0028] Figure 4 This is an example diagram of a bucket provided in Example 1 of the present invention.

[0029] Figure 5 This is an example diagram of a three-dimensional attribute provided in the first embodiment of the present invention.

[0030] Figure 6 It is a structural schematic diagram of a three-dimensional property detection device provided in Embodiment 2 of the present invention.

[0031] Figure 7 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] See also Figure 1 , shows a flow chart of a three-dimensional property detection method provided by the first embodiment of the present invention, the method can be performed by a three-dimensional property detection device, the three-dimensional property detection device can be implemented in the form of hardware and / or software, and the three-dimensional property detection device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0036] Step 101: In an elevator, a sensor located on the top of the car is called to detect downward and sense image data.

[0037] In different types of buildings, especially high-rise buildings, there are different transportation needs for people, pets, goods, etc. Therefore, different types of elevators can be deployed in the building according to different transportation needs, such as passenger elevators, freight elevators, sightseeing elevators, etc. The method in this embodiment can be applied to various types of elevators.

[0038] This embodiment can be applied to an elevator. An elevator is a comprehensive system. Different types of elevators have different structures.

[0039] In one example, a certain type of elevator is configured with: a controller (also known as an elevator control system), external call devices distributed on each floor, a car (including car doors), a motor for towing the car (also known as a traction machine), a control cabinet, a speed limiter, a door opener, a car frame, a car door, a counterweight guide rail, a car guide rail, a guide rail support, an accompanying cable, a counterweight device, a compensation chain (cable), a floor door, a guide device for the compensation chain (cable), a buffer, and the like.

[0040] Among them, the car door is equipped with a motor, which drives the car door to open and close.

[0041] In some types of elevators, the traction machine, control cabinet, speed limiter, traveling cable, etc. can be omitted.

[0042] These devices can be divided into different sets according to their functions to form various subsystems that support the operation of the elevator. The controller is connected to multiple systems of the elevator through serial ports or serial clock lines (SCL). The controller monitors each system and controls the operation of each subsystem, allowing the car to move in the shaft and reach each floor of the building.

[0043] In one example, the controller includes a door system, a frequency conversion system, a calling system and a traction system, wherein the door system is used to control the door of the elevator, the car is equipped with a car door, and the elevator lobby on each floor is provided with a hall door, and the elevator door includes a car door and a hall door of the elevator lobby on each floor, the type of the car door is the same as the type of the hall door, and they are switched at the same time (open or closed), the frequency conversion system is used to control the frequency converter, the calling system is used to control the logic of the internal call (calling the elevator in the car) and the external call (calling the elevator in the elevator lobby), and the traction system is used to control the movement of the car in the vertical direction (vertically upward or vertically downward) in the shaft.

[0044] The manager or owner of the elevator can choose whether to install an edge computing node on the elevator based on factors such as the load status of the elevator. If you choose not to install an edge computing node, the controller maintains the original control logic and does not affect the normal operation of the elevator. If you choose to install an edge computing node, you can select a suitable computing device as the edge computing node of the elevator according to your needs. The edge computing node is combined with the original controller into a new controller and redefines the control logic of the elevator.

[0045] Generally speaking, edge computing nodes are computing devices with strong computing capabilities, such as computers, servers or embedded devices. In addition, according to different intelligent services, edge computing nodes can be equipped with graphics processing units (GPUs) or embedded neural network processors (NPUs).

[0046] Edge computing nodes refer to a new business platform built at the edge of the network near the elevator, providing storage, computing, network and other resources, and sinking some key business applications to the edge of the access network to reduce the width and delay losses caused by network transmission and multi-level forwarding. The edge computing node is located between the user and the cloud (server), and is closer to the user (data source) than the traditional cloud. It has the characteristics of miniaturization, distribution and closer to the user. Massive data (such as audio data) no longer needs to be uploaded to the cloud for processing, and data processing is realized at the edge of the network, reducing request response time and network bandwidth while ensuring data security and privacy.

[0047] In addition, edge computing nodes can implement algorithm functions and model reasoning, communicate with existing controllers, and provide artificial intelligence (AI) and complex computing capabilities for existing controllers. Edge computing can also communicate with the cloud to implement algorithm functions and model updates, and transfer existing controller function calls and other functions.

[0048] One or more sensors are installed on the top of the elevator car. Figure 2 As shown, the sensor can be called to detect downward to obtain perceived image data.

[0049] In one case, the sensor is a depth sensor, and the depth sensor can collect depth values. In this case, the pixel points in the perceived image data have depth values.

[0050] In another case, the sensor is an infrared sensor, which can collect infrared values. In this case, the pixel points in the perceived image data also have infrared values.

[0051] Furthermore, the depth sensor and the infrared sensor may be configured in the same camera at the same time, or may be configured in different cameras, which is not limited in this embodiment.

[0052] The depth sensor and the infrared sensor have been jointly calibrated, and the conversion relationship (such as translation relationship, rotation relationship, etc.) between the coordinate system of the depth sensor and the coordinate system of the infrared sensor is calculated. The depth value is projected from the coordinate system of the depth sensor to the coordinate system of the infrared sensor based on the conversion relationship, or the infrared value is projected from the coordinate system of the infrared sensor to the coordinate system of the depth sensor based on the conversion relationship, so that the pixel points in the perceived image data have both depth values ​​and infrared values.

[0053] Step 102: self-correct the depth values ​​of the pixels in the perceived image data.

[0054] In practical applications, the depth values ​​of pixels in the perceived image data can be self-corrected, for example, by filtering out abnormal depth values, correcting depth values ​​with low accuracy, etc., to improve the quality of the depth values ​​of pixels, which is beneficial for subsequent recognition and processing.

[0055] In one embodiment of the present invention, when the pixel points in the perceived image data also have infrared values, considering that the accuracy of the distance of each pixel in the perceived image data is related to the strength of the return signal received by the sensor, the depth value can be corrected according to the infrared value.

[0056] Specifically, since the surface coverings and postures (relative to the viewing angle of the sensor surface) of objects or users in the elevator car are different, and the reflection characteristics of the inner wall of the car are relatively complex, after the sensor actively emits light to the surface of the object or user, the light returns to the sensor surface through multiple different paths and superimposes (multipath reflection), resulting in errors in the sensor's measurement of the distance to the object or user.

[0057] The accuracy of the distance measured by the sensor is related to the intensity of the received return light. The higher the light intensity, the more accurate the distance measurement. Even in the presence of multipath reflection, if the light intensity is higher, the main peak in the return light (usually formed by the return light of the shortest path) is relatively more significant, so the distance measurement is more accurate. On the contrary, there may be no significant main peak, that is, there is a large interference between the distances measured by different path returns, making it difficult to accurately measure the distance.

[0058] Since the sensor uses an active light source when detecting depth, and the infrared value can reflect the return light intensity of the corresponding pixel point, the depth value can be compensated based on the infrared value to correct the depth value of each pixel point and improve the ranging accuracy.

[0059] In this embodiment, step 102 may include the following steps:

[0060] Step 1021: Determine the neighborhood of the current pixel in the perceived image data.

[0061] In this embodiment, each pixel point in the perceived image data may be traversed in a certain order, and during the traversal process, a neighborhood may be divided for the current pixel point.

[0062] Exemplarily, taking the current pixel as the center, a square with a specified side length (such as 3×3, 5×5) is determined as its neighborhood.

[0063] Step 1022: Taking a preset reference value as a calibration standard for infrared values, the standard value is calculated using the depth values ​​of the pixels in the neighborhood.

[0064] In this embodiment, a reference value can be set for the infrared value by conducting experiments in an elevator car, etc., as a calibration standard, so that the depth values ​​of the pixels in the neighborhood can be statistically analyzed to obtain a standard value.

[0065] In a specific implementation, pixel points whose infrared values ​​are greater than a preset reference value may be screened in the neighborhood as target points, and the average value of the depth values ​​of the target points may be calculated as the standard value.

[0066] Pixels in the neighborhood whose infrared values ​​are less than or equal to the preset reference value can be ignored and no subsequent compensation is performed.

[0067] Step 1023: Subtract the depth value of the current pixel from the standard value to obtain a depth difference value.

[0068] For the current pixel, the depth value of the current pixel is subtracted from the standard value of other pixels in its neighborhood to obtain the depth difference value.

[0069] Step 1024: Generate a compensation coefficient according to the difference between the infrared value of the current pixel and the reference value.

[0070] In this embodiment, the difference between the infrared value of the current pixel and the reference value may be counted, and the compensation coefficient may be generated based on the difference between the infrared value of the current pixel and the reference value.

[0071] Generally, the compensation coefficient is positively correlated with the difference, that is, the greater the difference between the infrared value of the current pixel and the reference value, the greater the compensation coefficient, and conversely, the smaller the difference between the infrared value of the current pixel and the reference value, the smaller the compensation coefficient.

[0072] Exemplarily, the compensation coefficient is:

[0073]

[0074] Among them, ir (x,y) is the infrared value of the pixel (x, y), ir θ is the reference value, and max is the maximum value function.

[0075] In this example, when the infrared value is greater than or equal to the reference value, the difference between the infrared value of the current pixel and the reference value Less than or equal to 0, at this time, the depth value can be considered to have been accurately measured, no compensation is performed, and the compensation coefficient is 0; the difference between the infrared value of the current pixel and the reference value Gradually expand, the compensation coefficient will expand exponentially, the difference between the infrared value of the current pixel and the reference value When it reaches the maximum (i.e. the infrared value is 0), the compensation coefficient is 1.

[0076] Step 1025: multiply the depth difference value by the compensation coefficient to obtain a depth compensation value.

[0077] In this embodiment, the product of the depth difference value and the compensation coefficient may be calculated as the depth compensation value.

[0078] Step 1026: Add the depth compensation value to the depth value of the current pixel to complete the self-correction.

[0079] In this embodiment, a depth compensation value may be added to the depth value of the current pixel to obtain a new depth value, thereby completing self-correction of the depth value of the current pixel.

[0080] Exemplarily, the self-correction process is expressed as follows:

[0081]

[0082] Among them, depth (x,y) is the depth value of the pixel (x, y), ir (x,y) is the infrared value of the pixel (x, y), ir θ is the reference value, σ is the radius of the neighborhood, f(ir (x,y) ≥ir θ ) is the indicator function, in ir (x,y) ≥ir θ When f(ir (x,y) ≥ir θ )=1, in ir (x,y) <ir θ When f(ir (x,y) ≥ir θ )=0, max is the maximum value function.

[0083] Step 103: If the self-calibration is completed, the image data of the area where the target object is located is detected based on the perceived image data of the car.

[0084] In this embodiment, the target object can be set according to different services in the elevator, such as the user's head and shoulders, a robot, etc.

[0085] like Figure 3 As shown, when the sensed image data completes self-correction, the image data of the area where the target object is located can be detected in the sensed image data according to the characteristics of the car.

[0086] In a specific implementation, the target detection network can be trained in an offline environment based on historical perceived image data (especially perceived image data after linear changes in depth values) and image data of the area where the target object is located, which is annotated by the network. This allows the target detection network to have the ability to perform target detection in perceived image data with the target object as the target.

[0087] The target detection network is deployed in the elevator. During the operation of the elevator, the target detection network can be loaded and run.

[0088] Furthermore, the structure of the target detection network is not limited to artificially designed neural networks, but can also be a neural network optimized by a model quantization method, a neural network searched by a NAS (Neural Architecture Search) method, and so on. This embodiment does not impose any restrictions on this.

[0089] Among them, the target detection network can be divided into one-stage and two-stage.

[0090] Two-stage belongs to segment to segment, which means that the target detection operation is completed in two steps. The first step is to use various convolutional neural networks as the backbone of the target detection network to extract features of the perceived image data, perform coarse classification (distinguishing foreground and background) and coarse positioning (anchor) based on the features, and obtain candidate areas. The second step is to classify the candidate areas (i.e., target objects) in the classification network of the target detection network.

[0091] Exemplarily, the two-stage target detection operation may include R-CNN (Region-CNN, region-based convolutional neural network), Fast R-CNN (fast regional convolutional neural network), Faster R-CNN (faster regional convolutional neural network), R-FCN (Region-based fully convolutional network, region-based fully convolutional network), and the like.

[0092] One-stage is end-to-end, which means that the target detection operation is completed in one step. It does not search for candidate areas separately. The perceived image data is input into an overall network, and the generated detection results contain both the location and category information of the target object.

[0093] Exemplarily, the one-stage target detection operation may include SSD (Single Shot Multibox Detector), YOLO (You Only Look Once), and the like.

[0094] Generally speaking, two-stage detection has higher accuracy but slower detection speed, while one-stage detection has faster detection speed but lower accuracy. One-stage or two-stage can be selected based on factors such as elevator resources and real-time detection requirements, and this embodiment does not impose any restrictions on this.

[0095] For the perceived image data, the upper limit value of the sensor's detection in the car (that is, the distance from the pixel point on the sensor surface to the edge of the car) can be calculated based on the prior knowledge of the car. When there is a non-zero pixel point (that is, the effective detection distance) within the field of view and the depth value is greater than the upper limit value, it is considered to be beyond the detection range and the depth value of the pixel point is set to zero. This can filter out multipath interference or the depth background formed by objects.

[0096] In the specific implementation, a three-dimensional model of the car is constructed in the world coordinate system based on the specifications of the elevator (such as the door height of the car, the width of the car, the depth of the car, etc.), the installation parameters of the sensor (for example, when the sensor is installed in the middle of the car door light beam, the installation parameters include the left and right (x-axis) offset, front and back (y-axis) offset, inclination angle, etc. of the sensor relative to the lateral center point of the car door light), and the internal parameters of the sensor, and the three-dimensional model is inverted from the world coordinate system to the coordinate system of the sensor.

[0097] The point farthest from the sensor is measured in the three-dimensional model to obtain an upper limit value detected by the sensor in the car.

[0098] At this time, the depth value may be linearly changed according to the upper limit value, thereby standardizing the depth value.

[0099] Exemplarily, the linear change is expressed as:

[0100] val=(limit–dist)×G / limit;

[0101] Among them, val is the depth value after linear change, dist is the depth value before linear change, limit is the upper limit value, and G is the grayscale, such as 255.

[0102] In this example, the perceived image data may be converted into 8-bit data through linear transformation, that is, single-channel 8-bit 2-dimensional data with width and height being x and y respectively.

[0103] If the linear change is completed, the target object is set as the detection target for each frame of perceived image data, and the depth value is used as the pixel value. The perceived image data is input into the target detection network to detect the image data of the area where the target object is located.

[0104] Step 104: Under the condition of filtering out the car background, count the three-dimensional attributes of the target object in the regional image data.

[0105] In practical applications, based on prior knowledge of the elevator (such as the user's height, the robot's height, etc.), the background in the car can be adaptively filtered out in the regional image data to obtain the foreground representing the target object. In this case, the data in the regional image data (such as coordinates, depth values, etc.) are used to count the three-dimensional properties of the target object.

[0106] The background in the car may be existing facilities in the car (such as inner walls, etc.) or other target objects that have entered the regional image data.

[0107] In one embodiment of the present invention, step 104 may include the following steps:

[0108] Step 1041: In the regional image data, the pixels are sorted according to the depth values ​​to obtain a pixel sequence.

[0109] In the regional image data, for pixels with non-zero values, the pixels may be sorted according to the depth values ​​to obtain a pixel sequence.

[0110] like Figure 4 As shown, if the pixels are sorted from small to large according to the depth value, then in the pixel sequence, the closer to the sensor (that is, the higher the actual height), the higher the sorting.

[0111] Step 1042: Divide the pixel sequence into multiple buckets.

[0112] In this embodiment, the pixel sequence may be divided into a plurality of bins in an equal manner, and each bin has a plurality of pixels in a continuous order.

[0113] Step 1043: Calculate the gradient value of each bucket that changes on the overall depth value.

[0114] In this embodiment, the depth values ​​of the pixels in each bucket bin may be averaged to serve as the overall depth value of the bucket bin, thereby calculating the gradient value of each bucket bin in the overall depth value.

[0115] Step 1044: sort the gradient values ​​to obtain a gradient sequence.

[0116] In this embodiment, the gradient values ​​of each bin may be sorted from small to large or from large to small to obtain a gradient sequence.

[0117] Step 1045: Take a specified percentile value in the gradient sequence as a threshold.

[0118] In this embodiment, a specified quantile value (such as a binary value) is taken in the gradient sequence as a threshold for dividing the foreground and the background.

[0119] Step 1046: If the overall gradient value of the bucket is greater than or equal to the threshold, the pixel in the bucket is determined to be the background of the car.

[0120] If the overall gradient value of a bucket bin (such as the average value of the depth values ​​of the pixels in the bucket bin) is greater than or equal to the threshold, the pixels in the bucket bin are determined to be the background of the car and are ignored.

[0121] Step 1047: If the overall gradient value of the bucket is less than the threshold, the pixels in the bucket are used to calculate the three-dimensional properties of the target object.

[0122] like Figure 5As shown, if the overall gradient value of a bucket bin (such as the average value of the depth values ​​of the pixels in the bucket bin) is less than the threshold, the pixels in the bucket bin are determined to be the foreground of the car (i.e., the target object), and the pixels in these bucket bins are used to calculate the three-dimensional properties of the target object.

[0123] On the one hand, the average value of the pixel coordinates of the pixels in these bins is calculated as the coordinate anchor point, and the coordinate anchor point is projected into the world coordinate system to obtain the position of the target object.

[0124] On the other hand, the average depth value of the pixels in these bins is calculated as the distance anchor point, and the distance anchor point is projected into the world coordinate system to obtain the height of the target object.

[0125] In this embodiment, a sensor located at the top of the elevator car is called to detect downward the perceived image data; the pixels in the perceived image data have depth values; the depth values ​​of the pixels in the perceived image data are self-corrected; if the self-correction is completed, the regional image data where the target object is located is detected based on the car in the perceived image data; under the condition of filtering out the car background, the three-dimensional attributes of the target object are counted in the regional image data. This embodiment greatly reduces the amount of data through operations such as target detection and background filtering, thereby effectively reducing the amount of calculation, and is suitable for hardware with low computing power in elevators to improve its calculation efficiency. In addition, this embodiment gradually eliminates interference and gradually converges to the target object through operations such as target detection and background filtering, and counts the three-dimensional attributes of the target object, which can improve the accuracy of the three-dimensional attributes.

[0126] Embodiment 2

[0127] See also Figure 6 , shows a schematic diagram of the structure of a three-dimensional property detection device provided by Embodiment 2 of the present invention. Figure 6 As shown, the device comprises:

[0128] The car detection module 601 is used to call the sensor located on the top of the car in the elevator to detect downward the perceived image data; the pixel points in the perceived image data have depth values;

[0129] A depth correction module 602, configured to perform self-correction on the depth value of the pixel in the perceived image data;

[0130] The target detection module 603 is used to detect the regional image data where the target object is located in the sensed image data according to the car if the self-calibration is completed;

[0131] The attribute statistics module 604 is used to count the three-dimensional attributes of the target object in the regional image data under the condition of filtering out the car background.

[0132] In one embodiment of the present invention, the pixels in the perceived image data also have infrared values;

[0133] The depth correction module 602 includes:

[0134] A neighborhood determination module, used to determine the neighborhood of the current pixel point in the perceived image data;

[0135] A standard value calculation module, configured to calculate a standard value using a preset reference value as a calibration standard for the infrared value and using the depth value of the pixel point in the neighborhood;

[0136] A depth difference value calculation module, used for subtracting the depth value of the current pixel point from the standard value to obtain a depth difference value;

[0137] A compensation coefficient generating module, used for generating a compensation coefficient according to a difference between the infrared value of the current pixel point and the reference value;

[0138] A depth compensation value calculation module, used for multiplying the depth difference value by the compensation coefficient to obtain a depth compensation value;

[0139] The depth compensation value adding module is used to add the depth compensation value to the depth value of the current pixel point to complete self-correction.

[0140] In one embodiment of the present invention, the standard value calculation module includes:

[0141] A target point screening module, used for screening pixel points whose infrared values ​​are greater than a preset reference value in the neighborhood as target points;

[0142] The depth calculation module is used to calculate an average value of the depth values ​​of the target point as a standard value.

[0143] In one embodiment of the present invention, the compensation coefficient is:

[0144]

[0145] Among them, ir (x,y) is the infrared value of the pixel, ir θ is the reference value, and max is the maximum value function.

[0146] In one embodiment of the present invention, the target detection module 603 includes:

[0147] The target detection network loading module is used to load the target detection network;

[0148] An upper limit value calculation module, used for calculating the upper limit value detected by the sensor in the car;

[0149] A linear change module, used for linearly changing the depth value according to the upper limit value;

[0150] The image detection module is used to input the perceived image data into the target detection network to detect the image data of the area where the target object is located, taking the depth value as the pixel value if the linear change is completed.

[0151] In one embodiment of the present invention, the upper limit value calculation module includes:

[0152] A three-dimensional model building module, used to build a three-dimensional model of the car according to the specification parameters of the elevator, the installation parameters of the sensor, and the internal parameters of the sensor;

[0153] An upper limit value measurement module, used to measure the point farthest from the sensor in the three-dimensional model to obtain an upper limit value detected by the sensor in the car;

[0154] The linear change is expressed as:

[0155] val=(limit–dist)×G / limit;

[0156] Among them, val is the depth value after linear change, dist is the depth value before linear change, limit is the upper limit value, and G is the grayscale.

[0157] In one embodiment of the present invention, the attribute statistics module 604 includes:

[0158] A depth sorting module, used for sorting the pixel points in the regional image data according to the depth value to obtain a pixel sequence;

[0159] A bucket segmentation module, used for segmenting the pixel sequence into a plurality of buckets;

[0160] A gradient value calculation module, used for calculating the gradient value of each of the buckets that changes on the overall depth value;

[0161] A gradient sorting module, used for sorting the gradient values ​​to obtain a gradient sequence;

[0162] A threshold determination module, used for taking a specified quantile value in the gradient sequence as a threshold;

[0163] A background determination module, configured to determine that the pixel point in the bucket is the background of the car if the overall gradient value of the bucket is greater than or equal to the threshold;

[0164] The foreground statistics module is used to calculate the three-dimensional properties of the target object using the pixel points in the bucket if the overall gradient value of the bucket is less than the threshold.

[0165] In one embodiment of the present invention, the foreground statistics module includes:

[0166] A coordinate anchor point determination module, used to calculate an average value of pixel coordinates for the pixel points in the bucket as a coordinate anchor point;

[0167] A position projection module, used to project the coordinate anchor point into a world coordinate system to obtain the position of the target object;

[0168] A distance anchor point determination module, used to calculate an average value of the depth values ​​of the pixel points in the bucket as a distance anchor point;

[0169] The height projection module is used to project the distance anchor point into the world coordinate system to obtain the height of the target object.

[0170] The three-dimensional property detection device provided in the embodiment of the present invention can execute the three-dimensional property detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the three-dimensional property detection method.

[0171] Embodiment 3

[0172] See also Figure 7 , shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0173] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0174] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0175] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the detection method of three-dimensional attributes.

[0176] In some embodiments, the method for detecting three-dimensional properties may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for detecting three-dimensional properties described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the method for detecting three-dimensional properties in any other appropriate manner (e.g., by means of firmware).

[0177] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0178] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0179] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0181] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0182] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0183] Embodiment 4

[0184] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for detecting three-dimensional properties provided by any embodiment of the present invention is implemented.

[0185] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0186] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0187] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting three-dimensional properties, characterized in that: include: In an elevator, a sensor located on top of the car is used to detect and sense image data downwards; The pixels in the perceived image data have depth values; performing self-correction on the depth value of the pixel in the perceived image data; If the self-calibration is completed, then detecting the image data of the area where the target object is located in the sensed image data according to the car; Under the condition of filtering out the car background, the three-dimensional attributes of the target object are counted in the regional image data.

2. The method according to claim 1, characterized in that The pixels in the perceived image data also have infrared values; The self-correcting the depth value of the pixel in the perceived image data comprises: Determining a neighborhood of the current pixel in the perceived image data; Taking a preset reference value as a calibration standard for the infrared value, and using the depth value of the pixel point in the neighborhood to calculate a standard value; Subtract the depth value of the current pixel from the standard value to obtain a depth difference value; generating a compensation coefficient according to a difference between the infrared value of the current pixel point and the reference value; Multiplying the depth difference value by the compensation coefficient to obtain a depth compensation value; The depth compensation value is added to the depth value of the current pixel point to complete the self-correction.

3. The method according to claim 2, characterized in that The method of using a preset reference value as a calibration standard for the infrared value and using the depth value of the pixel point in the neighborhood to calculate the standard value includes: Selecting pixel points whose infrared values ​​are greater than a preset reference value in the neighborhood as target points; An average value is calculated for the depth values ​​of the target points as a standard value.

4. The method according to claim 2, characterized in that: The compensation coefficient is: Among them, ir (x,y) is the infrared value of the pixel, ir θ is the reference value, and max is the maximum value function.

5. The method according to claim 1, characterized in that: The detecting of the area image data where the target object is located in the perceived image data based on the car includes: Load the object detection network; Calculating an upper limit value detected by the sensor in the car; linearly changing the depth value according to the upper limit value; If the linear change is completed, the depth value is used as the pixel value, and the perceived image data is input into the target detection network to detect the image data of the area where the target object is located.

6. The method according to claim 5, characterized in that The calculating the upper limit value detected by the sensor in the car comprises: Constructing a three-dimensional model of the car according to the specification parameters of the elevator, the installation parameters of the sensor, and the internal parameters of the sensor; The point farthest from the sensor is measured in the three-dimensional model to obtain the upper limit value detected by the sensor in the car; the linear change is expressed as: val=(limit–dist)×G / limit; Among them, val is the depth value after linear change, dist is the depth value before linear change, limit is the upper limit value, and G is the grayscale.

7. The method according to any one of claims 1 to 6, characterized in that The step of counting the three-dimensional attributes of the target object in the regional image data under the condition of filtering out the car background comprises: In the regional image data, the pixel points are sorted according to the depth values ​​to obtain a pixel sequence; Dividing the pixel sequence into a plurality of buckets; Calculating, for each of the buckets, a gradient value that changes on the overall depth value; Sorting the gradient values ​​to obtain a gradient sequence; Taking a specified quantile value in the gradient sequence as a threshold; If the overall gradient value of the bucket is greater than or equal to the threshold, determining that the pixel point in the bucket is the background of the car; If the overall gradient value of the bucket is less than the threshold, the pixel points in the bucket are used to calculate the three-dimensional properties of the target object.

8. The method according to claim 7, characterized in that The using the pixel points in the bucket to calculate the three-dimensional attribute of the target object includes: Calculate the average value of pixel coordinates for the pixel points in the bucket as a coordinate anchor point; Projecting the coordinate anchor point into the world coordinate system to obtain the position of the target object; Calculate the average value of the depth values ​​for the pixel points in the bucket as a distance anchor point; The distance anchor point is projected into the world coordinate system to obtain the height of the target object.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the three-dimensional property detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting a three-dimensional attribute according to any one of claims 1 to 8 is implemented.

Images