Object identification method and device

Abstract

The invention provides an object recognition method and device. The object recognition method comprises the steps of obtaining a to-be-recognized depth image and preset key point information; according to the preset key point information, extracting depth information of at least two to-be-identified key points in a gate opening and closing area from the to-be-identified depth image; comparing the depth information of each to-be-identified key point with the preset key point information to obtain a key point comparison result of each to-be-identified key point; and identifying whether a target object exists in the gate opening and closing area or not according to a key point comparison result of the key points to be identified. On the basis of improving the accuracy of object recognition, the safety of the target object passing through the gate opening and closing area can be further improved, and pedestrians or other objects are prevented from being accidentally injured or damaged when the gate door is opened or closed.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an object recognition method. The present invention also relates to an object recognition device, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Urban rail transit such as subways and light rails has become an important way for urban residents to travel. During the peak period of inbound and outbound traffic, the efficiency of personnel traffic and the accuracy of traffic judgment of the gate affect people's traffic experience. The actions of the gate include opening, closing and emergency stop. Before opening or closing, the gate needs to detect whether there are other objects in the gate opening and closing area to prevent accidental injury to pedestrians or accidental damage to objects in the gate opening and closing area during the opening or closing process. In current practical applications, in gates that perform gate traffic logic judgment based on image recognition, it is usually determined whether there are pedestrians or other objects in the gate area by collecting images during the operation of the gate and using the target detection model to perform target detection on the image. However, this implementation method has the situation that the target detection model does not detect pedestrians or other objects in the image, but there are actually pedestrians or other objects. In this case, there is a risk of accidental injury to pedestrians or other objects when the gate is opened or closed. Summary of the invention

[0003] In view of this, an embodiment of this specification provides an object recognition method. One or more embodiments of this specification also relate to an object recognition device, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, there is provided an object recognition method, including: Obtain the depth image to be identified and the preset key point information; According to the preset key point information, extract the depth information of at least two key points to be identified in the gate opening and closing area in the depth image to be identified; The depth information of each key point to be identified is compared with the preset key point information to obtain a key point comparison result of each key point to be identified; According to the key point comparison results of each key point to be identified, it is identified whether there is a target object in the gate opening and closing area.

[0005] According to a second aspect of an embodiment of this specification, there is provided an object recognition device, including: An acquisition module is configured to acquire a depth image to be identified and preset key point information; An extraction module is configured to extract depth information of at least two key points to be identified in the gate opening and closing area in the depth image to be identified according to the preset key point information; A comparison module is configured to compare the depth information of each key point to be identified with the preset key point information to obtain a key point comparison result of each key point to be identified; The recognition module is configured to recognize whether there is a target object in the gate opening and closing area according to the key point comparison results of each key point to be recognized.

[0006] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above-mentioned object recognition method are implemented.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the above-mentioned object recognition method are implemented.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which implements the steps of the above-mentioned object recognition method when executed by a processor.

[0009] An embodiment of the present specification realizes that the gate opening and closing area is determined in the gate passage area, and multiple key points are pre-set in the gate opening and closing area, and the preset key point information of the multiple key points is obtained, so that in the actual operation of the gate, after the key points to be identified corresponding to the preset key points are extracted from the depth image to be identified, the depth information of each key point to be identified in the depth image to be identified is compared with the preset key point information, so as to determine whether there is a target object in the gate opening and closing area. By determining the gate opening and closing area in the gate passage area, the identification area for identifying the target object is reduced, and the accuracy of identifying the target object is improved on the basis of reducing the calculation workload; by obtaining the depth image to be identified and setting the preset key points for object identification, the detection error caused by using the target detection model to detect the image is avoided, and the accuracy of object identification is further improved; further, on the basis of improving the accuracy of object identification, the safety of the target object passing through the gate opening and closing area can be further improved, and pedestrians or other objects can be avoided from being accidentally injured when the gate door is opened or closed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1This is a schematic diagram of an application scenario of an object recognition method provided by an embodiment of this specification; Figure 2 is a flow chart of an object recognition method provided by one embodiment of this specification; Figure 3 It is a schematic diagram of a gate opening and closing area and preset key points provided by an embodiment of this specification; Figure 4 is a process flow chart of an object recognition method provided by an embodiment of this specification; Figure 5 is a schematic diagram of the structure of an object recognition device provided by an embodiment of this specification; Figure 6 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0011] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0012] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0014] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0015] First, the terms involved in one or more embodiments of this specification are explained.

[0016] Structured light camera: a 3D imaging device based on the principle of active stereo vision, which uses structured light technology to obtain three-dimensional geometric information of objects.

[0017] RGB image: It is a color image representation method based on the combination of three color channels: red, green, and blue. In an RGB image, the color of each pixel is determined by the intensity values ​​of these three channels, and the intensity range of each channel is usually 0 to 255. In digital image processing and computer graphics, the data of RGB images is usually organized into a three-dimensional array, where two dimensions represent the width and height of the image, and the third dimension represents the three color channels of red, green, and blue.

[0018] Urban rail transit such as subways and light rails has become an important way for urban residents to travel. During the peak period of inbound and outbound traffic, the efficiency of personnel traffic and the accuracy of traffic judgment of the gate affect people's traffic experience. The actions of the gate include opening, closing and emergency stop. Before opening or closing, the gate needs to detect whether there are other objects in the gate opening and closing area to prevent accidental injury to pedestrians or accidental damage to objects in the gate opening and closing area during the opening or closing process. In current practical applications, in gates that perform gate traffic logic judgment based on image recognition, it is usually determined whether there are pedestrians or other objects in the gate area by collecting images during the operation of the gate and using the target detection model to perform target detection on the image. However, this implementation method has the situation that the target detection model does not detect pedestrians or other objects in the image, but there are actually pedestrians or other objects. In this case, there is a risk of accidental injury to pedestrians or other objects when the gate is opened or closed.

[0019] In this specification, an object recognition method is provided. This specification also relates to an object recognition apparatus, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0020] See also Figure 1, Figure 1 The following is a schematic diagram of an application scenario of an object recognition method provided according to an embodiment of the present specification. Figure 1 As shown, the passenger is passing through the gate passage area along the gate passage direction. In actual applications, before entering the gate, the passenger needs to swipe the card in the gate sensing area to open the gate door. After the gate door is opened, the passenger can pass through the gate. In this process, before opening or closing the gate door, the gate door needs to identify whether there are pedestrians or other objects in the gate passage area to avoid accidental injury to pedestrians or damage to objects. However, when the passenger passes through the gate, misidentification may occur, resulting in accidental injury to pedestrians or damage to objects in the gate opening and closing area.

[0021] In actual applications, if a passenger is accidentally injured by the gate, it means that the passenger is currently in the operating range that can be reached during the actual operation of the gate, that is, the gate opening and closing area. Therefore, in order to improve the safety of passengers when passing through the gate passage area, that is, to improve the accurate identification of passengers in the gate passage area during the actual operation of the gate, this manual can specifically obtain the depth image to be identified and the preset key point information during the actual operation of the gate, accurately determine the gate opening and closing area in the depth image to be identified through the preset key point information, and extract the key points to be identified corresponding to the preset key points and the depth information of the key points to be identified in the gate opening and closing area of ​​the depth image to be identified. The depth information of each extracted key point to be identified is further compared with the preset key point information, so as to identify whether there is a target object (passenger or object) in the gate opening and closing area after obtaining the key point comparison results of each key point to be identified.

[0022] This specification provides an embodiment in which a depth image to be identified is obtained, and the depth image to be identified is processed in the subsequent object recognition process. There is no need to collect color images for target detection, thereby improving the accuracy of target detection using a target detection model; the gate opening and closing area is determined in the depth image to be identified by presetting key point information, and the identification area used for object identification is reduced, thereby reducing the workload of subsequent object recognition; after extracting the depth information of the key points to be identified, the depth information of each key point to be identified is compared with the preset key point information, and the key point comparison result is used to identify whether there is a target object in the gate opening and closing area, thereby improving the accuracy of object recognition in the gate opening and closing area.

[0023] It should be noted that the above-mentioned subway gate application scenario is only used to explain the object recognition method provided in this specification. The object recognition method provided in this specification is not limited to the subway gate application scenario, but can also be applied to smart automatic door scenarios, access control gate scenarios, etc.

[0024] See also Figure 2 , Figure 2 A flowchart of an object recognition method provided according to an embodiment of the present specification is shown, which specifically includes the following steps: Step 202: Obtain a depth image to be identified and preset key point information.

[0025] The depth image to be identified refers to the gate depth image acquired by the image acquisition device during the actual operation of the gate. The image acquisition device may be a structured light camera (specifically a monocular structured light camera or a binocular structured light camera) or other depth cameras.

[0026] Preferably, in one or more embodiments provided in this specification, the image acquisition device adopts a top-down perspective, that is, the device lens faces the ground, and the line connecting the physical center of the camera lens and the lens focus is perpendicular to the ground. Based on this, the depth image to be identified acquired by the image acquisition device is also a gate depth image of the top-down perspective.

[0027] Since the gate opening and closing area cannot be determined in the acquired depth image to be identified, the gate opening and closing area can be determined in the depth image to be identified by obtaining preset key point information. The preset key point information specifically includes pixel position information and depth range information of at least two preset key points determined in advance. The pixel position information can specifically be the coordinate position information of the preset key point in the pixel coordinate system; the depth range information refers to the depth range corresponding to the preset key point. The depth range information can be one or more. If the depth range information is one, multiple preset key points correspond to the same depth range information. If the depth range information is multiple, the number is the same as the number of preset key points.

[0028] Since the preset key point information is the key to perform object recognition on the depth image to be recognized, it is crucial to determine and generate the preset key point information before performing object recognition during the actual operation of the gate.

[0029] In a specific implementation manner provided in this specification, before obtaining the depth image to be identified and the preset key point information, the method further includes: Acquire a static color image of a target gate and a static depth image corresponding to the static color image; Determining a gate opening and closing area of ​​the target gate and at least two preset key points of the gate opening and closing area in the static color image; Determine color space position information of the at least two preset key points, and determine depth space position information of the at least two preset key points in the static depth image according to the color space position information; Determining pixel position information and pixel depth information of the at least two preset key points according to the depth space position information; The preset key point information of the at least two preset key points is generated according to the pixel position information and the pixel depth information.

[0030] Among them, the target gate refers to any gate in the gate passage area where there is no target object, or any gate that is not in operation. The static color image refers to a static color image of the gate passage area of ​​the target gate where there is no target object or is not in operation. The static depth image refers to a static depth image of the gate passage area of ​​the target gate where there is no target object or is not in operation. The static color image and the static depth image are acquired at the same time.

[0031] The gate opening and closing area specifically refers to the operating range that the gate can reach during actual operation. Color space position information refers to the coordinate position information of the color pixel point in the color camera space coordinate system, specifically the coordinate position information of the preset key point in the color camera space coordinate system. Depth space position information refers to the coordinate position information of the depth pixel point in the depth camera space coordinate system, specifically the coordinate position information of the preset key point in the depth camera space coordinate system. Pixel position information refers to the coordinate position information of the depth pixel point in the pixel coordinate system, specifically the coordinate position information of the preset key point in the pixel coordinate system. Pixel depth information refers to the depth value of the depth pixel point, specifically the depth value of the preset key point.

[0032] Specifically, a color camera is used to obtain a static color image of the target gate's passage area in which the target object does not exist or is not in operation, and a depth camera is used to obtain a static depth image of the target gate's passage area in which the target object does not exist or is not in operation. The gate opening and closing area of ​​the target gate is identified in the static color image, and a plurality of preset key points in the gate opening and closing area are determined. In actual applications, the preset key points can be obtained by manual annotation, or by training a key point annotation model. The key point annotation model can be specifically used to annotate a plurality of preset key points in a static color image. The method for obtaining the preset key points can be specifically determined according to the actual application situation, and this specification does not limit it here.

[0033] After obtaining multiple preset key points in a static color image, it is necessary to determine the coordinate position information of each preset key point in the color camera space coordinate system, that is, the color space position information. Specifically, the color space position information of the preset key points can be determined by solving the intrinsic and extrinsic parameters of the color camera.

[0034] Since the object recognition method provided in this specification is based on the depth image to be identified to identify the target object, it is necessary to further determine the pixel position information of multiple preset key points. Specifically, the depth space position information of the preset key points in the static depth image can be determined based on the determined color space position information, and then the depth space position information of the preset key points can be used to calculate the pixel position information of the preset key points, and determine the pixel depth information corresponding to the preset key points. Specifically, the depth space position information of the preset key points in the static depth image can be determined according to the following formula 1: Formula 1 in, is the depth space position information, is the color space position information, is the rotation transformation matrix, is the translation vector. The depth space position information of each preset key point can be calculated based on the color space position information of each preset key point through the above formula 1. In practical applications, the pixel position information of the preset key point can be calculated based on the conversion relationship between the depth camera space coordinate system and the pixel coordinate system.

[0035] Further, see Figure 3 , Figure 3 A schematic diagram of a gate opening and closing area and preset key points provided according to an embodiment of this specification is shown. Figure 3 As shown, the dotted line frame area is specifically the static color image of the target gate, and the solid line frame area is specifically the area where the target gate is located. Figure 3 The shaded area shown in the figure is the gate passage area of ​​the target gate. Figure 3 The marked area shown in the figure is the gate opening and closing area of ​​the target gate. Points A, B, C, D, E, F, G, H, and K are all preset key points, and the coordinate position information of each preset key point in the color camera space coordinate system, that is, the color space position information, are A (x1, y4), B ((x1+x2) / 2, y4), C (x2, y4), D (x1, (y3+y4) / 2), E ((x1+x2) / 2, (y3+y4) / 2), F (x2, (y3+y4) / 2), G (x1, y3), H ((x1+x2) / 2, y3), and K (x2, y3). After determining the color space position information of each preset key point, the pixel position information and pixel depth information of each preset key point can be determined based on the above method, so as to generate the preset key point information of the preset key point according to the pixel position information and the pixel depth information in the subsequent process.

[0036] It should be noted that the number of preset key points can be adjusted according to actual application conditions, for example, 9, 12, 15, etc., and the arrangement of the preset key points can also be adjusted according to actual application conditions, for example, rectangular, cross-shaped, etc.

[0037] In a specific implementation manner provided in this specification, generating preset key point information of the at least two preset key points according to the pixel position information and the pixel depth information includes: Determine a target key point among the at least two preset key points, and determine target pixel depth information of the target key point among the pixel depth information; Calculating target static mean features and target static difference features of the target key points according to the target pixel depth information; Generating target depth range information of the target key point according to the target static mean feature and the target static difference feature; The target pixel position information of the target key point is determined in the pixel position information, and the target pixel position information and the target depth range information are determined as preset key point information of the target key point.

[0038] Among them, the target key point refers to any one of the preset key points. The target pixel depth information refers to the pixel depth information of the target key point. The target static mean feature refers to the depth mean feature of the target key point, specifically the mean of the depth values ​​of the target key point in each static depth image. The target static difference feature refers to the depth standard deviation feature of the target key point, specifically the standard deviation of the depth values ​​of the target key point in each static depth image. The target depth range information refers to the depth range information of the target key point. The target pixel position information refers to the pixel position information of the target key point.

[0039] Specifically, taking any one of the preset key points (i.e., the target key point) as an example, the target pixel depth information of the target key point is determined in the pixel depth information of each preset key point, and the target static mean feature of the target key point is calculated according to the target pixel depth information. The calculation method is specifically referred to the following formula 2: Formula 2 in, is the target static mean feature, j The value of is determined according to the number of preset key points. is the number of static depth images and static color images, For the j The preset key point is in i Depth information in a static depth image.

[0040] In practical applications, n groups of static color images and static depth images are collected, and the static mean features of each preset key point are calculated by the above formula 2. The number of static color images and static depth images can be determined according to the actual application situation, for example ≥1000, etc., which is not limited in this specification.

[0041] Still taking any one of the preset key points as an example, after calculating and obtaining the target static mean feature of the target key point, the target static difference feature of the target key point can be calculated according to the target static mean feature, as shown in the following formula 3: Formula 3 Wherein, S is the target static difference feature. The target static difference feature of the target key point is calculated based on the above formula 3. Since there is no target object in the static depth image, the depth information corresponding to each preset key point is the floor depth value of the gate opening and closing area. Since the bottom of the gate door in the gate opening and closing area is often a certain distance away from the floor, it is not necessary to consider the depth range from the bottom of the gate door to the floor in the subsequent object recognition process. Therefore, in order to further improve the accuracy of subsequent object recognition, the depth range from the bottom of the gate door to the floor can be ignored.

[0042] Based on this, after obtaining the target static mean feature and target static difference feature of the target key point, the distance between the bottom of the gate and the floor of the gate opening and closing area is obtained, so as to generate the target depth range information of the target key point according to the target static mean feature, the target static difference feature and the distance between the bottom of the gate and the floor of the gate opening and closing area. The target depth range information can be specifically expressed as ,in, It is the distance between the bottom of the gate and the floor of the gate opening and closing area. The value range of the target depth range information can also be fine-tuned according to the depth information distribution of each preset key point.

[0043] The target pixel position information of the target key point is determined in the pixel position information of each preset key point, and the target pixel position information and the above-determined target depth range information are the preset key point information of the target key point.

[0044] The above is a process for generating the preset key point information of any preset key point among the preset key points. The preset key point information of other preset key points is also generated based on the above method to obtain the preset key point information of each preset key point. Therefore, during the actual operation of the gate, the object recognition of the gate opening and closing area can be performed using the depth image to be recognized based on the preset key point information of each preset key point.

[0045] This specification provides one or more embodiments, in which multiple groups of static color images and static depth images are obtained, the static color images are used to determine the gate opening and closing area, and multiple preset key points and color space position information of each preset key point are determined in the static color image. According to the color space position information, the depth space position information of each preset key point in the static depth image is determined, the pixel position information and pixel depth information of each preset key point are determined according to the depth space position information, and after the static mean feature and static difference feature of each preset key point are generated according to the pixel depth information, the depth range information of each preset key point is generated, so that the pixel position information and depth range information are determined as the preset key point information of each preset key point. The gate opening and closing area is determined based on the static color image, and the preset key point information of each preset key point is determined using the static depth image, so that in the subsequent process, the corresponding key point to be identified is determined in the depth image to be identified according to the preset key point information, thereby completing the object recognition process.

[0046] Step 204: extracting depth information of at least two key points to be identified in the gate opening and closing area from the depth image to be identified according to the preset key point information.

[0047] After obtaining the depth image to be identified and the preset key point information, since the gate opening and closing area cannot be determined based on the depth image to be identified alone, the gate opening and closing area of ​​the gate can be determined in the depth image to be identified according to the preset key point information, specifically, the key points in the gate opening and closing area are determined.

[0048] Among them, the key points to be identified refer to the key points extracted from the depth image to be identified. After extracting multiple key points to be identified from the depth image to be identified, the depth information of each key point to be identified is obtained. The implementation method of extracting the key points to be identified based on the preset key point information and the depth information of the key points to be identified is as follows: In a specific implementation provided in this specification, according to the preset key point information, extracting the depth information of at least two key points to be identified in the gate opening and closing area in the depth image to be identified includes: Acquire target pixel position information of a target key point in the preset key point information, wherein the target key point is any one of the preset key points; According to the target pixel position information, key points to be processed are determined in the depth image to be identified, and depth information to be processed of the key points to be processed is extracted.

[0049] The key point to be processed refers to the key point to be identified that has the same target pixel position information as the target key point. The depth information to be processed refers to the depth information of the key point to be processed.

[0050] Specifically, any one of the preset key points (target key point) is still used as an example for explanation, and the target pixel position information of the target key point is obtained in the preset key point information, and according to the target pixel position information, the key point to be identified in the depth image to be identified that is the same as the target pixel position information is determined as the key point to be processed corresponding to the target key point, and the depth information to be processed of the key point to be processed is extracted. According to the same method, the key points to be identified corresponding to each preset key point and the depth information of each key point to be identified are respectively extracted in the depth image to be identified. For example, the preset key points specifically include points A, B, C, D, E, F, G, H, and K, then according to the pixel position information of each point A, B, C, D, E, F, G, H, and K, the corresponding A', B', C', D', E', F', G', H', and K' can be determined in the depth image to be identified.

[0051] In one embodiment provided in this specification, the pixel position information of each preset key point in the preset key point information is used to extract the key points to be identified in the depth image to be identified, and the depth information of each key point to be identified is determined. Thus, in the actual operation of the gate, it is only necessary to obtain the depth image to be identified of the gate to determine the gate opening and closing area of ​​the gate and the key points to be identified in the gate opening and closing area in the depth image to be identified, without the need to obtain a color image to determine the gate opening and closing area, thereby avoiding erroneous identification of objects based on color images, improving the accuracy of subsequent object identification, and reducing the computational workload.

[0052] Step 206: compare the depth information of each key point to be identified with the preset key point information to obtain a key point comparison result of each key point to be identified.

[0053] The key points to be identified are extracted from the depth image to be identified, and the depth information of the key points to be identified is determined. The depth information of the key points to be identified can be compared with the preset key point information to obtain the key point comparison result of the key points to be identified. In practical applications, the depth information of each key point to be identified is compared with the preset key point information, specifically, the depth information of each key point to be identified is compared with the depth range information in the preset key point information, so as to determine whether the depth information of each key point to be identified is within the depth range information.

[0054] Based on this, in a specific implementation manner provided in this specification, the depth information of each key point to be identified is respectively compared with the preset key point information to obtain the key point comparison result of each key point to be identified, including: Acquire target depth range information of the target key point in the preset key point information; It is determined whether the depth information to be processed is within the target depth range information, and a comparison result of the key point to be processed is obtained.

[0055] The key point comparison result to be processed refers to the key point comparison result of the key point to be processed.

[0056] Specifically, taking any one of the key points to be identified (i.e., the key point to be processed) as an example, the target depth range information of the target key point corresponding to the key point to be processed is obtained in the preset key point information, and it is determined whether the depth information of the key point to be processed (i.e., the depth information to be processed) is within the target depth range information, and the comparison result of the key point to be processed is obtained.

[0057] For example, the depth information of the key point A' to be identified is , the target depth range information of the preset key point A is ,judge Is it located in Within, a key point comparison result of the key point A' to be identified is generated.

[0058] In addition, in order to reduce the tediousness of subsequent object recognition in the gate opening and closing area, the unified depth range information corresponding to each preset key point can also be generated according to the static mean feature and static difference feature of each preset key point, and the depth information of each key point to be identified is compared with the unified depth range information, and the key point comparison results corresponding to each key point to be identified are generated. The method of generating the depth range information and the method of using the depth range information to compare the depth information of each key point to be identified can be determined according to the actual application situation, and this manual does not limit it here.

[0059] An embodiment provided in the present specification realizes a comparison of the depth information of each key point to be identified in the depth image to be identified with the depth range information of its corresponding preset key point to generate a key point comparison result. In a subsequent process, it can be determined whether there is a target object in the gate opening and closing area based on the key point comparison result, thereby improving the accuracy of object recognition in the gate opening and closing area.

[0060] Step 208: According to the key point comparison results of each key point to be identified, it is identified whether there is a target object in the gate opening and closing area.

[0061] After obtaining the key point comparison results of each key point to be identified, object recognition can be performed on the gate opening and closing area based on the key point comparison results of each key point to be identified to identify whether there is a target object in the gate opening and closing area. The target object can specifically be a pedestrian, an object, etc.

[0062] Since the number of preset key points is often large, and since the key points to be identified correspond to the preset key points one by one, the key points to be identified extracted from the depth image to be identified will also be large. In order to ensure the accuracy of object recognition, object recognition can be performed on the gate opening and closing areas in turn according to the key point comparison results of each key point to be identified, as follows: In a specific implementation manner provided in this specification, according to the key point comparison results of each key point to be identified, identifying whether there is a target object in the gate opening and closing area includes: When the depth information to be processed is within the target depth range information, it is determined that there is a target object in the gate opening and closing area.

[0063] Specifically, for any one of the key points to be identified (i.e., the key point to be processed), if the depth information to be processed of the key point to be processed is within the target depth range information of the target key point corresponding thereto, it is determined that there is a target object in the gate opening and closing area. If the depth information to be processed of the key point to be processed is not within the target depth range information, it can be determined whether the depth information of the next key point to be identified is within the depth range information of the preset key point corresponding thereto, so as to determine whether there is a target object in the gate opening and closing area.

[0064] Based on this, in a specific implementation manner provided in this specification, the method further includes: In the case where the depth information to be processed is not located in the target depth range information, determining a new key point to be processed as a key point to be traversed among the key points to be identified, and obtaining a comparison result of the key points to be traversed of the key points to be traversed; When the to-be-traversed depth information of the to-be-traversed key point is within the to-be-traversed depth range information, determining that there is a target object in the gate opening and closing area; When the to-be-traversed depth information is not within the to-be-traversed depth range information, return to the step of determining a new to-be-processed key point as a to-be-traversed key point among the to-be-identified key points, and obtaining a to-be-traversed key point comparison result of the to-be-traversed key point, until the traversal of each to-be-identified key point is completed.

[0065] Specifically, if the depth information to be processed is not within the target depth range information, a new key point to be processed is determined as a key point to be traversed (i.e., the next key point to be identified) in each key point to be identified, and the comparison result of the key point to be traversed is obtained. If the depth information to be traversed of the key point to be traversed is within the depth range information to be traversed of the corresponding preset key point, it is determined that there is a target object in the gate opening and closing area. If the depth information to be traversed of the key point to be traversed is not within the depth range information to be traversed, a new key point to be processed is determined again until all key points to be identified are traversed. If the depth information of each key point to be identified is not within the depth range information corresponding to it, it is determined that there is no target object in the gate opening and closing area.

[0066] Furthermore, in order to improve the efficiency of object recognition in the gate opening and closing area, the key point comparison results of each key point to be identified can also be obtained at the same time, and whether there is a target object in the gate opening and closing area is identified based on the key point comparison results of each key point to be identified.

[0067] Based on this, in another specific implementation manner provided in this specification, according to the key point comparison results of each key point to be identified, identifying whether there is a target object in the gate opening and closing area includes: In the key point comparison results of each key point to be identified, it is determined whether there is depth information of the key point to be identified that is located in the depth range information corresponding to the key point to be identified; When the depth information of the key point to be identified is located in the depth range information corresponding to the key point, it is determined that the target object exists in the gate opening and closing area.

[0068] Specifically, after obtaining the key point comparison results of each key point to be identified, it is determined in the key point comparison results of each key point to be identified whether there is depth information of the key point to be identified within the corresponding depth range information; if there is depth information of the key point to be identified within the corresponding depth range information, it is determined that there is a target object in the opening and closing area of ​​the gate.

[0069] In one or more embodiments provided in this specification, the key point comparison results of each key point to be identified can be used in turn to identify whether a target object exists in the gate opening and closing area, or the key point comparison results of each key point to be identified can be obtained simultaneously. In the key point comparison results of each key point to be identified, it is determined whether there is depth information of the key point to be identified that is located in the depth range information corresponding to it, to identify whether a target object exists in the gate opening and closing area. By using multiple object recognition methods, it is possible to identify whether a target object exists in the gate opening and closing area, thereby improving the flexibility and accuracy of object recognition.

[0070] The object recognition method provided in this specification includes: obtaining a depth image to be identified and preset key point information; extracting depth information of at least two key points to be identified in the gate opening and closing area in the depth image to be identified based on the preset key point information; comparing the depth information of each key point to be identified with the preset key point information respectively to obtain a key point comparison result of each key point to be identified; and identifying whether there is a target object in the gate opening and closing area based on the key point comparison result of each key point to be identified.

[0071] One or more embodiments of the present specification realize that the gate opening and closing area is determined in the gate passage area, and multiple key points are pre-set in the gate opening and closing area, and the preset key point information of the multiple key points is obtained, so that in the actual operation of the gate, the depth image to be identified is collected, and the key points to be identified corresponding to the preset key points are extracted from the depth image to be identified, and the depth information of each key point to be identified in the depth image to be identified is compared with the preset key point information, so as to determine whether there is a target object in the gate opening and closing area. By determining the gate opening and closing area in the gate passage area, the identification area for identifying the target object is reduced, and the accuracy of identifying the target object is improved on the basis of reducing the calculation workload; by obtaining the depth image to be identified and setting the preset key points for object identification, the detection error caused by using the target detection model to detect the image is avoided, and the accuracy of object identification is further improved; further, on the basis of improving the accuracy of object identification, the safety of the target object passing through the gate opening and closing area can be further improved, and pedestrians or other objects can be avoided from being accidentally injured when the gate door is opened or closed.

[0072] The following combination Figure 4 , taking the application of the object recognition method provided in this specification in the subway gate scene as an example, the object recognition method is further explained. Figure 4 A flowchart of a processing process of an object recognition method provided by an embodiment of this specification is shown. Figure 4 As shown, the object recognition method provided in this specification can be specifically divided into a pre-processing stage and an actual processing stage.

[0073] First, the pre-processing stage is explained. A static color image of the subway gate is obtained through a color camera, and a static depth image of the subway gate is obtained through a depth camera. Multiple preset key points are determined in the static color image, and the color space position information of each preset key point in the color camera space coordinate system is calculated. According to the color space position information of each preset key point, the depth space position information of each preset key point in the depth camera space coordinate system is determined. Through the conversion relationship between the color camera space coordinate system and the depth camera space coordinate system, the pixel position information and pixel depth information of each preset key point are determined, thereby realizing data alignment between the static color image and the static depth image. Then, according to the pixel depth information of each preset key point, the static mean feature and static difference feature of each preset key point are calculated, and after obtaining the distance between the bottom of the subway gate door and the floor of the gate opening and closing area, the depth range information of each preset key point is generated according to the static mean feature and the static difference feature, which can be specifically expressed as .

[0074] It should be noted that the above-mentioned pre-processing stage is implemented when there are no passengers passing through the subway gate. The pre-processing stage does not need to be executed again when the subway gate is actually in operation. The pixel position information and depth range information of each preset key point can be directly obtained for use.

[0075] Next, the actual processing stage is explained. The depth image to be identified of the subway gate during the actual operation is obtained by the depth camera, and the key points to be identified corresponding to each preset key point and the depth information of each key point to be identified are extracted from the depth image to be identified according to the pixel position information of each preset key point. The depth information of each key point to be identified is compared with the depth range information of the preset key point corresponding to it to determine whether the depth information of the key point to be identified is located within the depth range information of the preset key point corresponding to it. If, among the key points to be identified, there is any key point to be identified whose depth information is located within the depth range information of the preset key point corresponding to it, it can be determined that there are passengers or other passing objects in the gate opening and closing area of ​​the subway gate, and the gate opening and closing area is in an unsafe state. If, among the key points to be identified, there is no key point to be identified whose depth information is located within the depth range information of the preset key point corresponding to it, it can be determined that there are no passengers or other passing objects in the gate opening and closing area of ​​the subway gate, and the gate opening and closing area is in a safe state.

[0076] In one embodiment of the present specification, when there are no passengers passing through the subway gate in advance, by acquiring a static color image and a static depth image of the subway gate, multiple key points are set in the gate opening and closing area, and preset key point information of multiple key points is generated, so that in the actual operation of the gate, by acquiring a depth image to be identified, the key points to be identified corresponding to the preset key points are extracted from the depth image to be identified, and the depth information of each key point to be identified in the depth image to be identified is compared with the depth range information, so as to determine whether there are passengers or other passing objects in the gate opening and closing area. By determining the gate opening and closing area in the gate passing area, the identification area is reduced, and the accuracy of identifying passengers or other passing objects is improved on the basis of reducing the calculation workload; by acquiring the depth image to be identified and setting the preset key points for object identification, the detection error caused by using the target detection model to detect the image is avoided, and the accuracy of object identification is further improved; further, on the basis of improving the accuracy of object identification, the safety of passengers or other passing objects passing through the gate opening and closing area can be further improved, and it is avoided that the gate door is opened or closed to accidentally injure passengers or damage other passing objects.

[0077] Corresponding to the above method embodiment, this specification also provides an object recognition device embodiment, Figure 5 FIG. 1 shows a schematic diagram of the structure of an object recognition device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises: An acquisition module 502 is configured to acquire a depth image to be identified and preset key point information; An extraction module 504 is configured to extract depth information of at least two key points to be identified in the gate opening and closing area in the depth image to be identified according to the preset key point information; The comparison module 506 is configured to compare the depth information of each key point to be identified with the preset key point information to obtain a key point comparison result of each key point to be identified; The identification module 508 is configured to identify whether there is a target object in the gate opening and closing area according to the key point comparison results of each key point to be identified.

[0078] Optionally, the device further includes a generating module configured to: Acquire a static color image of a target gate and a static depth image corresponding to the static color image; Determining a gate opening and closing area of ​​the target gate and at least two preset key points of the gate opening and closing area in the static color image; Determine color space position information of the at least two preset key points, and determine depth space position information of the at least two preset key points in the static depth image according to the color space position information; Determining pixel position information and pixel depth information of the at least two preset key points according to the depth space position information; The preset key point information of the at least two preset key points is generated according to the pixel position information and the pixel depth information.

[0079] Optionally, the generating module is further configured to: Determine a target key point among the at least two preset key points, and determine target pixel depth information of the target key point among the pixel depth information; Calculating target static mean features and target static difference features of the target key points according to the target pixel depth information; Generating target depth range information of the target key point according to the target static mean feature and the target static difference feature; The target pixel position information of the target key point is determined in the pixel position information, and the target pixel position information and the target depth range information are determined as preset key point information of the target key point.

[0080] Optionally, the extraction module 504 is further configured to: Acquire target pixel position information of a target key point in the preset key point information, wherein the target key point is any one of the preset key points; According to the target pixel position information, key points to be processed are determined in the depth image to be identified, and depth information to be processed of the key points to be processed is extracted.

[0081] Optionally, the comparison module 506 is further configured to: Acquire target depth range information of the target key point in the preset key point information; It is determined whether the depth information to be processed is within the target depth range information, and a comparison result of the key point to be processed is obtained.

[0082] Optionally, the identification module 508 is further configured to: When the depth information to be processed is within the target depth range information, it is determined that there is a target object in the gate opening and closing area.

[0083] Optionally, the device further includes a traversal module configured to: In the case where the depth information to be processed is not located in the target depth range information, determining a new key point to be processed as a key point to be traversed among the key points to be identified, and obtaining a comparison result of the key points to be traversed of the key points to be traversed; When the to-be-traversed depth information of the to-be-traversed key point is within the to-be-traversed depth range information, determining that there is a target object in the gate opening and closing area; When the to-be-traversed depth information is not within the to-be-traversed depth range information, return to the step of determining a new to-be-processed key point as a to-be-traversed key point among the to-be-identified key points, and obtaining a to-be-traversed key point comparison result of the to-be-traversed key point, until the traversal of each to-be-identified key point is completed.

[0084] Optionally, the identification module 508 is further configured to: In the key point comparison results of each key point to be identified, it is determined whether there is depth information of the key point to be identified that is located in the depth range information corresponding to the key point to be identified; When the depth information of the key point to be identified is located in the depth range information corresponding to the key point, it is determined that the target object exists in the gate opening and closing area.

[0085] The object recognition device provided in this specification includes: an acquisition module, which is configured to acquire a depth image to be identified and preset key point information; an extraction module, which is configured to extract depth information of at least two key points to be identified in the gate opening and closing area from the depth image to be identified according to the preset key point information; a comparison module, which is coordinated to compare the depth information of each key point to be identified with the preset key point information respectively to obtain a key point comparison result of each key point to be identified; and a recognition module, which is configured to identify whether there is a target object in the gate opening and closing area according to the key point comparison result of each key point to be identified.

[0086] One or more embodiments of the present specification realize that the gate opening and closing area is determined in the gate passage area, and multiple key points are pre-set in the gate opening and closing area, and the preset key point information of the multiple key points is obtained, so that in the actual operation of the gate, the depth image to be identified is collected, and the key points to be identified corresponding to the preset key points are extracted from the depth image to be identified, and the depth information of each key point to be identified in the depth image to be identified is compared with the preset key point information, so as to determine whether there is a target object in the gate opening and closing area. By determining the gate opening and closing area in the gate passage area, the identification area for identifying the target object is reduced, and the accuracy of identifying the target object is improved on the basis of reducing the calculation workload; by obtaining the depth image to be identified and setting the preset key points for object identification, the detection error caused by using the target detection model to detect the image is avoided, and the accuracy of object identification is further improved; further, on the basis of improving the accuracy of object identification, the safety of the target object passing through the gate opening and closing area can be further improved, and pedestrians or other objects can be avoided from being accidentally injured when the gate door is opened or closed.

[0087] The above is a schematic scheme of an object recognition device of this embodiment. It should be noted that the technical scheme of the object recognition device and the technical scheme of the object recognition method described above are of the same concept, and the details not described in detail in the technical scheme of the object recognition device can be referred to the description of the technical scheme of the object recognition method described above.

[0088] Figure 6 The structure block diagram of a computing device 600 provided according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0089] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0090] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0091] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 600 may also be a mobile or stationary server.

[0092] The processor 620 implements the steps of the object recognition method when executing the computer program / instructions.

[0093] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above object recognition method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above object recognition method.

[0094] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of the object recognition method described above are implemented.

[0095] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the object recognition method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the object recognition method described above.

[0096] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned object recognition method when executed by a processor.

[0097] The above is a schematic solution of a computer program product of this embodiment. It should be noted that the technical solution of the computer program product and the technical solution of the object recognition method described above are of the same concept, and the details not described in detail in the technical solution of the computer program product can be found in the description of the technical solution of the object recognition method described above.

[0098] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] The computer program / instruction includes a computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0100] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0101] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0102] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An object recognition method, characterized in that: include: Obtain the depth image to be identified and the preset key point information; According to the preset key point information, extract the depth information of at least two key points to be identified in the gate opening and closing area in the depth image to be identified; The depth information of each key point to be identified is compared with the preset key point information to obtain a key point comparison result of each key point to be identified; According to the key point comparison results of each key point to be identified, it is identified whether there is a target object in the gate opening and closing area.

2. The method according to claim 1, characterized in that Before obtaining the depth image to be identified and the preset key point information, the method further includes: Acquire a static color image of a target gate and a static depth image corresponding to the static color image; Determining a gate opening and closing area of ​​the target gate and at least two preset key points of the gate opening and closing area in the static color image; Determine color space position information of the at least two preset key points, and determine depth space position information of the at least two preset key points in the static depth image according to the color space position information; Determining pixel position information and pixel depth information of the at least two preset key points according to the depth space position information; The preset key point information of the at least two preset key points is generated according to the pixel position information and the pixel depth information.

3. The method according to claim 2, characterized in that Generating preset key point information of the at least two preset key points according to the pixel position information and the pixel depth information, comprising: Determine a target key point among the at least two preset key points, and determine target pixel depth information of the target key point among the pixel depth information; Calculating target static mean features and target static difference features of the target key points according to the target pixel depth information; Generating target depth range information of the target key point according to the target static mean feature and the target static difference feature; The target pixel position information of the target key point is determined in the pixel position information, and the target pixel position information and the target depth range information are determined as preset key point information of the target key point.

4. The method according to claim 1, characterized in that According to the preset key point information, the depth information of at least two key points to be identified in the gate opening and closing area is extracted from the depth image to be identified, including: Acquire target pixel position information of a target key point in the preset key point information, wherein the target key point is any one of the preset key points; According to the target pixel position information, key points to be processed are determined in the depth image to be identified, and depth information to be processed of the key points to be processed is extracted.

5. The method according to claim 4, characterized in that The depth information of each key point to be identified is compared with the preset key point information to obtain the key point comparison result of each key point to be identified, including: Acquire target depth range information of the target key point in the preset key point information; It is determined whether the depth information to be processed is within the target depth range information, and a comparison result of the key point to be processed is obtained.

6. The method according to claim 5, characterized in that According to the key point comparison results of each key point to be identified, identifying whether there is a target object in the gate opening and closing area includes: When the depth information to be processed is within the target depth range information, it is determined that there is a target object in the gate opening and closing area.

7. The method according to claim 6, characterized in that The method further comprises: In the case where the depth information to be processed is not located in the target depth range information, determining a new key point to be processed as a key point to be traversed among the key points to be identified, and obtaining a comparison result of the key points to be traversed of the key points to be traversed; When the to-be-traversed depth information of the to-be-traversed key point is within the to-be-traversed depth range information, determining that there is a target object in the gate opening and closing area; When the to-be-traversed depth information is not within the to-be-traversed depth range information, return to the step of determining a new to-be-processed key point as a to-be-traversed key point among the to-be-identified key points, and obtaining a to-be-traversed key point comparison result of the to-be-traversed key point, until the traversal of each to-be-identified key point is completed.

8. The method according to claim 5, characterized in that According to the key point comparison results of each key point to be identified, identifying whether there is a target object in the gate opening and closing area includes: In the key point comparison results of each key point to be identified, it is determined whether there is depth information of the key point to be identified that is located in the depth range information corresponding to the key point to be identified; When the depth information of the key point to be identified is located in the depth range information corresponding to the key point, it is determined that the target object exists in the gate opening and closing area.

9. An object recognition device, characterized in that: include: An acquisition module is configured to acquire a depth image to be identified and preset key point information; An extraction module is configured to extract depth information of at least two key points to be identified in the gate opening and closing area in the depth image to be identified according to the preset key point information; A comparison module is configured to compare the depth information of each key point to be identified with the preset key point information to obtain a key point comparison result of each key point to be identified; The recognition module is configured to recognize whether there is a target object in the gate opening and closing area according to the key point comparison results of each key point to be recognized.

10. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. It is characterized in that when the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium storing a computer program / instruction, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images