Object recognition method, device and system and computer readable storage medium

Abstract

The invention relates to an object recognition method, device and system, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a video stream image, and extracting object features of a to-be-recognized object in the video stream image; matching the object features with standard features of the custom object to obtain a matching result; wherein the standard features are determined by performing three-dimensional reconstruction on the object information of the custom object; and if the matching result is that the matching is successful, marking the user-defined object in the video stream image. By adopting the method, the use scene of the camera equipment can be expanded.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an object recognition method, device, system, computer-readable storage medium, and computer program product. Background Art

[0002] Camera equipment (such as webcam), also known as computer camera, computer eye, electronic eye, etc., is a video input device that is widely used in video conferencing and telemedicine. Generally, camera equipment can obtain environmental images by real-time monitoring of the surrounding environment, so that users can see corresponding people, cars, animals, etc. from the environmental images.

[0003] However, current camera devices can often only identify preset objects (such as people, cars, and animals, etc.), which limits the use scenarios of the camera devices. Summary of the invention

[0004] Based on this, in order to address the technical problem that current camera devices can only recognize preset objects (such as people, cars, and animals, etc.) but cannot recognize customized objects, which limits the usage scenarios of the camera devices, an object recognition method, device, system, computer-readable storage medium and computer program product are provided that can expand the usage scenarios of the camera devices.

[0005] In a first aspect, the present application provides an object recognition method. The method comprises:

[0006] Acquire a video stream image, and extract object features of an object to be identified in the video stream image;

[0007] Matching the object features with the standard features of the custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of the object information of the custom object;

[0008] If the matching result is a successful match, the custom object is marked in the video stream image.

[0009] In one embodiment, the construction method of the standard feature includes:

[0010] Reconstructing the custom object in three dimensions according to the object information to obtain a three-dimensional image, and outputting the three-dimensional image to a user terminal;

[0011] When a confirmation operation of the user terminal on the three-dimensional image is received, standard features of the custom object are obtained according to the object information.

[0012] In one embodiment, the method further comprises:

[0013] receiving a modification operation of the user terminal on the object information, and obtaining the modified object information and a modified three-dimensional image corresponding to the modified object information;

[0014] When a confirmation operation of the user terminal on the modified three-dimensional image is received, the standard feature is obtained according to the modified object information.

[0015] In one embodiment, the object information of the custom object is obtained by at least one of the following methods:

[0016] The first one:

[0017] Receiving images of the custom object at different viewing angles and text information of the images at different viewing angles uploaded by a user terminal; the object information of the custom object includes the images of the custom object at different viewing angles and text information corresponding to the images at different viewing angles;

[0018] Second type:

[0019] The video image uploaded by the user terminal and the video description corresponding to the video image are received; the object information of the custom object includes the video image of the custom object and the video description corresponding to the video image.

[0020] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features;

[0021] The matching of the object features with the standard features of the custom object to obtain a matching result includes:

[0022] When the similarity between the object image feature and the standard image feature is greater than or equal to an image similarity threshold, and the similarity between the object text feature and the standard text feature is greater than or equal to a text similarity threshold, the matching result is determined to be a successful match.

[0023] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; the method further includes:

[0024] When the similarity between the object image feature and the standard image feature is greater than or equal to the image similarity threshold, and the similarity between the object text feature and the standard text feature is less than the text similarity threshold, outputting the object image of the object to be identified and the text information of the object to be confirmed to the user terminal;

[0025] When a confirmation operation of the user terminal on the object image of the object to be identified and the text information of the object to be confirmed is received, it is determined that the matching result is a successful match.

[0026] In one embodiment, the method further comprises:

[0027] When receiving a modification operation of the user terminal on the text information of the object to be confirmed and a confirmation operation on the object image of the object to be identified, determining that the matching result is a successful match;

[0028] Obtaining modified object text information and modified object text features corresponding to the modified object text information;

[0029] According to the modified object text feature and the standard text feature, a modified standard text feature is obtained.

[0030] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; the method further includes:

[0031] When the similarity between the object image feature and the standard image feature is less than an image similarity threshold, and the similarity between the object text feature and the standard text feature is less than a text similarity threshold, the matching result is determined to be a matching failure.

[0032] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; the method further includes:

[0033] When the similarity between the object image feature and the standard image feature is less than an image similarity threshold, and the similarity between the object text feature and the standard text feature is greater than or equal to a text similarity threshold, the matching result is determined to be a matching failure.

[0034] In a second aspect, the present application further provides an object recognition device, the device comprising:

[0035] An acquisition module, used to acquire a video stream image and extract object features of an object to be identified in the video stream image;

[0036] A processing module, used for matching the object features with the standard features of the custom object to obtain a matching result; wherein the standard features are determined by three-dimensionally reconstructing the object information of the custom object;

[0037] The marking module is used to mark the custom object in the video stream image if the matching result is a successful match.

[0038] In a third aspect, the present application also provides an object recognition system, which includes a camera device and a processor; the camera device is used to acquire video stream images and transmit the video stream images to the processor; the processor is used to extract object features of an object to be recognized in the video stream images; the object features are matched with standard features of a custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of the object information of the custom object; if the matching result is a successful match, the custom object is marked in the video stream image.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0040] Acquire a video stream image, and extract object features of an object to be identified in the video stream image;

[0041] Matching the object features with the standard features of the custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of the object information of the custom object;

[0042] If the matching result is a successful match, the custom object is marked in the video stream image.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Acquire a video stream image, and extract object features of an object to be identified in the video stream image;

[0045] Matching the object features with the standard features of the custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of the object information of the custom object;

[0046] If the matching result is a successful match, the custom object is marked in the video stream image.

[0047] The object recognition method, device, camera device, computer-readable storage medium and computer program product described above, by acquiring a video stream image, extracting the object features of the object to be recognized in the video stream image, and matching the object features with the standard features of the custom object, because the standard features are determined by three-dimensional reconstruction of the object information of the custom object, by three-dimensional reconstruction of the custom object, the reconstructed custom object can be approximated to the real custom object, thereby improving the accuracy of the standard features of the custom object. Thus, when the object features are matched with the standard features of the custom object, and the matching result is a successful match, it can be determined that there is a custom object in the video stream image, so that the camera device can not only recognize the preset objects (such as people, cars and animals, etc.) based on the original function, but also recognize the user-defined custom objects, thereby expanding the use scenarios of the camera device. Moreover, by marking the custom object in the video stream image, the user can quickly recognize the custom object from the video stream image, thereby improving the recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A diagram of an application environment of an object recognition method in an embodiment;

[0049] Figure 2 is a schematic flow chart of an object recognition method in one embodiment;

[0050] Figure 3 A schematic diagram of a flow chart of a method for constructing standard features of a custom object in one embodiment;

[0051] Figure 4 is a flow chart of an object recognition method in another embodiment;

[0052] Figure 5 A schematic diagram of a flow chart of a method for obtaining object information of a custom object in one embodiment;

[0053] Figure 6 A schematic diagram of obtaining standard features of a custom object in another embodiment;

[0054] Figure 7 is a flow chart of an object recognition method in another embodiment;

[0055] Figure 8 is a flow chart of an object recognition method in another embodiment;

[0056] Fig. 9 is a flow chart of an object recognition method in another embodiment;

[0057] Fig.10 FIG. 4 is a structural block diagram of an object recognition device in an embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] The object recognition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The object recognition system 100 includes a camera device 102 and a processor 104. The camera device 102 communicates with the processor 104 via a network. The camera device 102 may include a camera or other types of devices for real-time monitoring. The processor 104 may be a device deployed at the local edge or a remote cloud device. Specifically, the camera device 102 is used to obtain video stream images and transmit video recorded images to the processor 104. The processor 104 obtains the video stream image, extracts the object features of the object to be identified in the video stream image; matches the object features with the standard features of the custom object to obtain a matching result, and when the matching result is a successful match, marks the custom object in the video stream image. The processor 104 also communicates with the user terminal 106 via the network, and the user can upload the object information of the custom object through the user terminal 106.

[0060] Specifically, in one embodiment, Figure 2 As shown, an object recognition method is provided, which is applied to Figure 1 Taking the processor 104 in the example as an example, the following steps may be included:

[0061] S202, acquiring a video stream image, and extracting object features of an object to be identified in the video stream image.

[0062] In this embodiment, the video stream image is a real-time image monitored by the camera device, and the camera device can transmit the video stream image to the processor. The object can refer to an object with vital signs and / or an object without vital signs. For example, an object with vital signs can include people and animals, and an object without vital signs can include vehicles and tables. The object to be identified is an object with vital signs and / or an object without vital signs in the video stream image. The number of objects to be identified is at least one. In other words, at least one actual object in the video stream image can be used as an object to be identified.

[0063] In this embodiment, the object features of the object to be identified are used to identify the object to be identified, and the object features of the object to be identified include the object image features of the object to be identified and the object text features corresponding to the object image features. The object image features of the object to be identified are used to identify the image of the object to be identified, and the object text features are used to identify the purpose and name of the object to be identified. Specifically, the object image features of the object to be identified can be obtained by processing the video stream image through the visual model, and the text information corresponding to the video stream image can be obtained by inputting the object image features into the image understanding model, and the text information corresponding to the video stream image can be obtained by inputting the text information corresponding to the video stream image into the language model, and the object text features can be obtained.

[0064] Among them, the visual model is a computer vision technology that analyzes and processes images or videos to extract feature information, thereby realizing the understanding and recognition of images or videos. The visual model can be regarded as a mathematical model that can convert pixels in an image or video into feature vectors with semantic information, so as to better describe and express the content and structure of the image or video. The core of the visual model is feature extraction, which is the process of converting information in an image or video into a feature vector with semantic information. The visual model can extract some representative features from an image or video, such as edges, corners, textures, etc., to realize the understanding and recognition of an image or video. The application of visual models is very extensive, among which the most common applications are image recognition and target detection. In image recognition, the visual model can convert features in an image into feature vectors, and then use machine learning algorithms to classify these feature vectors to realize image recognition. For example, in face recognition, a visual model can be used to convert features in a face image into feature vectors, and then use machine learning algorithms to classify these feature vectors to realize face recognition.

[0065] Among them, the language model is a natural language processing technology. It analyzes and processes texts to establish a probabilistic model of language, thereby realizing the understanding and generation of texts. The language model can be regarded as a mathematical model that can convert words or character sequences in texts into vectors with probabilistic information, so as to better describe and express the language rules and structure of texts. The core of the language model is the probabilistic model, which establishes a probabilistic model of language by counting and analyzing words or character sequences in texts. These algorithms can extract some representative features from texts, such as the frequency of occurrence of words or character sequences, contextual information, etc., so as to realize the understanding and generation of texts. The application of language models is very extensive, among which the most common applications are speech recognition, machine translation and natural language generation. In speech recognition, language models can convert speech signals into texts, and then use machine learning algorithms to recognize the texts. In natural language generation, language models can generate texts that conform to language rules and structures based on the input semantic information.

[0066] S204, matching the object features with the standard features of the custom object to obtain a matching result.

[0067] In this embodiment, the custom object refers to an object that is user-defined and can be identified in a video stream image. The object can be an object with vital signs and / or an object without vital signs. The standard features of the custom object are used to identify the custom object, and the standard features of the custom object are determined by three-dimensional reconstruction of the object information of the custom object. Specifically, the standard features of the custom object can be obtained based on the object information of the custom object. The object information of the custom object includes an image of the custom object and text information corresponding to the image. The image of the custom object is used to identify the actual appearance of the custom object, and the text information corresponding to the image is used to identify the name and purpose of the custom object.

[0068] By processing the image of the custom object through the visual model, the standard image features of the custom object can be obtained. By processing the text information corresponding to the image through the language model, the standard text features corresponding to the standard image features can be obtained. The standard features of the custom object include the standard image features of the custom object and the standard text features corresponding to the standard image features.

[0069] Among them, 3D reconstruction is a process of converting a 2D image or video into a 3D model through computer technology. It is a technology widely used in computer vision, computer graphics, robotics, medical imaging and other fields. The 3D reconstruction process requires the use of some computer vision and computer graphics technologies. Through 3D reconstruction, the image or video data of the plane can be converted into a 3D model with depth and stereoscopic sense, so as to better understand and process these data. The steps of 3D reconstruction usually include data acquisition, preprocessing, feature extraction, matching, triangulation, surface reconstruction, etc. First, it is necessary to collect 2D image or video data, and then preprocess the data, such as denoising, enhancement, etc. Next, feature extraction and matching are required to find similar points in the image or video. Then, these points are connected by triangulation to form a triangular mesh. Finally, the triangular mesh is converted into a 3D model with depth and stereoscopic sense through surface reconstruction.

[0070] In this embodiment, the matching result refers to the result after the object to be identified is matched with the custom object, and the matching result may include a successful match or a failed match. Specifically, when the object to be identified is a custom object, the matching result is a successful match; when the object to be identified is not a custom object, the matching result is a failed match.

[0071] S206: If the matching result is successful, mark the custom object in the video stream image.

[0072] In this embodiment, when the matching result is a successful match, it means that the object to be identified is a custom object. By marking the custom object in the video stream image, the user can quickly identify the custom object from the video stream image, thereby improving the recognition efficiency.

[0073] In summary, based on Figure 2The method shown, by acquiring a video stream image, extracting the object features of the object to be identified in the video stream image, and matching the object features with the standard features of the custom object, because the standard features are determined by three-dimensional reconstruction of the object information of the custom object, by three-dimensional reconstruction of the custom object, the reconstructed custom object can be approximated to the real custom object, thereby improving the accuracy of the standard features of the custom object. Thus, when the object features are matched with the standard features of the custom object, and the matching result is a successful match, it can be determined that there is a custom object in the video stream image, so that the camera device can not only recognize the preset objects (such as people, cars and animals, etc.) based on the original function, but also recognize the user-defined custom objects, which expands the use scenario of the camera device. Moreover, by marking the custom object in the video stream image, the user can quickly identify the custom object from the video stream image, thereby improving the recognition efficiency. Moreover, compared with the existing situation that before using the recognition model to recognize the object, a large number of object pictures need to be collected and the recognition model needs to be retrained, which causes extremely high economic costs and application complexity, the method of the present application does not need to collect a large number of pictures of custom objects, and can use the existing visual model and language model for recognition processing, thereby reducing the economic cost.

[0074] The standard features of the custom object can be constructed in various possible ways, as long as the object to be identified in the video stream image can be recognized based on the constructed standard features.

[0075] In one embodiment, Figure 3 As shown, a flowchart of a method for constructing a standard feature of a custom object is provided, and this method is applied to Figure 1 Taking the processor 104 in the example as an example, the following steps may be included:

[0076] S302: Perform three-dimensional reconstruction on the custom object according to the object information to obtain a three-dimensional image, and output the three-dimensional image to a user terminal.

[0077] In this embodiment, the user terminal may refer to a terminal that can display an application software (Application, APP) or a Web (World Wide Web) page for controlling the monitoring area of ​​the camera device, and a terminal that can view video stream images and display three-dimensional images. The user can adjust the monitoring area of ​​the camera device through the APP, and when the monitoring area is different, the video stream image collected by the camera device is also different.

[0078] In this embodiment, the user can intuitively and vividly see the custom object through the three-dimensional image, so that the user can adjust the object information of the custom object conveniently, so that the adjusted three-dimensional image can be closer to the real custom object.

[0079] S304: When a confirmation operation of the user terminal on the three-dimensional image is received, standard features of the custom object are obtained according to the object information.

[0080] In this embodiment, the user confirms the three-dimensional image through the user terminal, which means that the three-dimensional image is close to the real custom object, and further indicates that the object information of the custom object used when reconstructing the three-dimensional image is accurate, so that the standard features of the custom object can be obtained based on the object information.

[0081] In summary, based on Figure 3 The method shown can obtain the standard features of the custom object, thereby matching the object features of the object to be identified in the video stream image with the standard features of the custom object. Since the standard features are determined based on the three-dimensional reconstruction of the object information of the custom object, the reconstructed custom object can be approximated to the real custom object by three-dimensionally reconstructing the custom object, thereby improving the accuracy of the standard features of the custom object. Thus, when the object features are matched with the standard features of the custom object and the matching result is a successful match, it can be determined that there is a custom object in the video stream image. Thus, the camera device can not only identify the preset objects (such as people, cars, and animals, etc.) based on the original function, but also identify the user-defined custom objects, thereby expanding the use scenarios of the camera device. Moreover, by marking the custom object in the video stream image, the user can quickly identify the custom object from the video stream image, thereby improving the recognition efficiency.

[0082] In one embodiment, Figure 4 As shown, an object recognition method is provided, which is applied to Figure 1 Taking the processor 104 in the example as an example, the following steps may be included:

[0083] S402: Receive a modification operation of the user terminal on the object information, and obtain the modified object information and a modified three-dimensional image corresponding to the modified object information.

[0084] In this embodiment, when the user submits a request to modify the object information through the user terminal, it indicates that the three-dimensional image of the custom object is not close to the real custom object. The user modifies the object information of the custom object through the user terminal, and the processor can obtain the modified object information, and then the processor performs three-dimensional reconstruction of the custom object according to the modified object information, obtains the modified three-dimensional image, and re-outputs the modified three-dimensional image to the user terminal.

[0085] S404: when a confirmation operation of the user terminal on the modified three-dimensional image is received, a standard feature is obtained according to the modified object information.

[0086] In this embodiment, the user confirms the modified three-dimensional image through the user terminal, which means that the modified three-dimensional image is close to the real custom object, and further indicates that the modified object information used when reconstructing the modified three-dimensional image is accurate, so that the standard features of the custom object can be obtained based on the modified object information.

[0087] In summary, based on Figure 4 The method shown can adjust the three-dimensional image of the object to be identified by receiving modification feedback from the user when the reconstructed three-dimensional image is inaccurate. Therefore, when the user terminal receives a confirmation operation on the modified three-dimensional image, it indicates that the modified object information when reconstructing the modified three-dimensional image is accurate. Therefore, the standard features of the custom object can be obtained based on the modified object information, thereby improving the accuracy of the standard features of the custom object. Therefore, when the object is identified based on the standard features of the custom object, the recognition accuracy can be improved.

[0088] The object information of the custom object can be obtained in various possible ways, as long as the custom object can be reconstructed based on the object information of the component.

[0089] In one embodiment, Figure 5 As shown, a flowchart of a method for obtaining object information of a custom object is provided, and the method is applied to Figure 1 Taking the processor 104 in the example as an example, the following steps may be included:

[0090] S502: Receive images of a custom object at different viewing angles and text information of the images at different viewing angles uploaded by a user terminal.

[0091] In this embodiment, the images of the custom object at different perspectives and the text information of the images at different perspectives can be uploaded through the APP or Web page. Specifically, when the user uploads the images of the custom object at different perspectives and the text information of the images at different perspectives through the APP or Web page, the processor can receive the images of the custom object at different perspectives and the text information of the images at different perspectives uploaded by the user terminal in response to the user's triggering operation of the upload button.

[0092] Specifically, the object information of the custom object includes images of the custom object at different perspectives and text information corresponding to the images at different perspectives. From the user's perspective, the object images at different perspectives may refer to images of the custom object in the front, back, left, right, top, and other directions. From the object's perspective, the object images at different perspectives may refer to images of the custom object at different angles. Thus, through the object images at different perspectives, the custom object can be reconstructed in all directions, so that the reconstructed three-dimensional image is close to the real custom object.

[0093] Specifically, the text information of images at different perspectives refers to information that introduces images at different perspectives, and the text information of images at different perspectives may include the direction of the custom object in the image, the name, shape, and purpose of the custom object, etc. For example, if the custom object is a table, and the images at different perspectives are images of the table in the front direction, the corresponding text information may include: table (i.e., the name of the custom object), front direction (i.e., the direction of the custom object in the image), and used to place things (i.e., the purpose of the custom object).

[0094] Combined with the contents described in S502, it can be known that the processor can obtain the object information of the custom object by uploading images of the custom object at different perspectives and text information of the images at different perspectives through the user terminal. In some embodiments, the processor can also obtain the object information of the custom object by uploading a video image of the custom object. The former method can be referred to as the method in which the processor obtains the object information of the custom object in a predetermined mode, and the latter method can be referred to as the method in which the processor obtains the object information of the custom object in a free mode. Specifically, the specific implementation process of the processor obtaining the object information of the custom object in the free mode is shown in S504 and S506 below.

[0095] S504: Receive a video image uploaded by a user terminal and a video description corresponding to the video image.

[0096] In this embodiment, the object information of the custom object includes a video image of the custom object and a video description corresponding to the video image. The video image is an image of the custom object taken at different perspectives, and the video description corresponding to the video object refers to the description information of the custom object in the video image. The video description corresponding to the video object may include the name, shape, and purpose of the custom object. The video image may be a video with a preset time, and the video description may be description information with a preset number of characters. In this embodiment, the preset time may be 10 seconds to 20 seconds, and the preset number of characters may be 20 to 100.

[0097] Specifically, when a user uploads a video image on an APP or a Web page, the user can enter a video description corresponding to the video image on the APP or the Web page. In response to the user triggering an upload button on the APP or the Web page, the processor can receive the video image uploaded by the user terminal and the video description corresponding to the video image.

[0098] In summary, Figure 5 In the method shown, two different ways are provided to upload the object information of the custom object, so that different types of users can choose a suitable way to upload the object information of the custom object according to their own situations, thereby improving the selectivity of information uploading.

[0099] Combination Figure 1-Figure 5 The content shown, such as Figure 6 As shown, a schematic diagram for obtaining standard features of a custom object is provided, wherein a user terminal displays an initialization interface on an APP or a Web page, and the initialization interface is used to display a predetermined mode and a free mode to the user.

[0100] Specifically, when the user selects a predetermined mode, the user terminal jumps to the corresponding interface, and the user can upload the image of the custom object on the interface, such as the image of the custom object in five perspectives: front, back, left, right, and top, and upload the text description of the image of the custom object at the same time. When the user triggers the upload button on the interface, the processor responds to the user's triggering operation on the upload button and can receive the image of the custom object and the text description of the image uploaded by the user terminal. Thus, the processor can reconstruct the custom object in three dimensions based on the image of the custom object and the text description of the image, and output the three-dimensional image to the user terminal. When receiving the confirmation operation of the user terminal on the three-dimensional image, the processor processes the image of the custom object at different perspectives through the visual model, and can obtain the standard image features of the custom object. The text information of the image at different perspectives is processed through the language model, and the standard text features corresponding to the standard image features can be obtained, so that the processor stores the standard image features and standard text features of the custom object (i.e., the standard features of the custom object) in the database for subsequent use in object recognition.

[0101] Specifically, when the user selects the free mode, the user terminal jumps to the corresponding interface, the user can upload a 10-20 second video image on the interface, and the user can enter a 201-00 character video description on the interface. When the user triggers the upload button on the interface, the processor responds to the user's triggering operation on the upload button, and can receive the video image uploaded by the user terminal and the video description corresponding to the video image. Thus, the processor can reconstruct the custom object according to the video image and the video description corresponding to the video image, obtain a three-dimensional image, and output the three-dimensional image to the user terminal. When receiving the confirmation operation of the user terminal on the three-dimensional image, the processor processes the video image through the visual model to obtain the standard image features of the custom object, and processes the video description corresponding to the video image through the language model to obtain the standard text features corresponding to the standard image features, so that the processor stores the standard image features and standard text features of the custom object in the database for subsequent use in object recognition.

[0102] In the contents of the predetermined mode and free mode described above, if the user has not confirmed the three-dimensional image, the user can adjust the images of the custom object in the front, back, left, right, and top directions and the text information of the images from different perspectives according to actual conditions, or adjust the video image and the video description corresponding to the video image until the user terminal receives confirmation of the modified three-dimensional image.

[0103] The processor can achieve matching of the object features of the object to be identified with the standard features of the custom object in various possible ways to obtain a matching result, as long as the object to be identified can be accurately identified.

[0104] In one embodiment, the object features of the object to be identified include the object image features of the object to be identified and the object text features corresponding to the object image features, and the standard features of the custom object include the standard image features of the custom object and the standard text features corresponding to the standard image features. Specifically, the object features are matched with the standard features of the custom object to obtain a matching result, including: when the similarity between the object image features and the standard image features is greater than or equal to the image similarity threshold, and the similarity between the object text features and the standard text features is greater than or equal to the text similarity threshold, determining that the matching result is a successful match. Thus, by marking the custom object in the video stream image, the user can quickly identify the custom object from the video stream image, thereby improving the recognition efficiency.

[0105] In one embodiment, when the similarity between the object image feature and the standard image feature is less than the image similarity threshold, and the similarity between the object text feature and the standard text feature is less than the text similarity threshold, the matching result is determined to be a matching failure. Alternatively, when the similarity between the object image feature and the standard image feature is less than the image similarity threshold, and the similarity between the object text feature and the standard text feature is greater than or equal to the text similarity threshold, the matching result is determined to be a matching failure. Thus, the above two methods can simultaneously determine that there is no custom object in the video stream image, and no marking process is performed in the video stream image.

[0106] In one embodiment, the object features include the object image features of the object to be identified and the object text features corresponding to the object image features, and the standard features include the standard image features and the standard text features corresponding to the standard image features; Figure 7 As shown, an object recognition method is provided, which is applied to Figure 1 Taking the processor 104 in the example as an example, the following steps may be included:

[0107] S702, when the similarity between the object image feature and the standard image feature is greater than or equal to the image similarity threshold, and the similarity between the object text feature and the standard text feature is less than the text similarity threshold, output the object image of the object to be identified and the object text information to be confirmed to the user terminal.

[0108] In this embodiment, the output object image of the object to be identified may be a picture of the object to be identified captured from the video stream image, or may be a video of the object to be identified captured from the video stream image, which is not specifically limited in this embodiment.

[0109] In this embodiment, the text information of the object to be confirmed is obtained according to the object image features of the object to be identified. Specifically, the object image features of the object to be identified are input into the image understanding model to obtain the text information of the object to be confirmed.

[0110] S704: When receiving a confirmation operation of the user terminal on the object image to be identified and the text information of the object to be confirmed, determining that the matching result is a successful match.

[0111] In summary, based on Figure 7 When the method shown determines that the matching result is a successful match, it means that the recognition result of the object to be recognized is accurate, that is, the object to be recognized is a custom object. Therefore, by marking the custom object in the video stream image, the user can quickly recognize the custom object from the video stream image, thereby improving the recognition efficiency. Moreover, compared with the existing method of only using the image information of the object for object recognition, the present application also uses the text information of the object, thereby solving the existing misrecognition situation and improving the recognition accuracy.

[0112] Understandably, Figure 7 It describes the situation where the user confirms the object image of the object to be identified and the text information of the object to be confirmed. Other possible situations include: the user confirms the object image of the object to be identified but does not confirm the text information of the object to be confirmed; or the user confirms neither the object image of the object to be identified nor the text information of the object to be confirmed.

[0113] It can be understood that when the user confirms the object image of the object to be identified but does not confirm the text information of the object to be identified, it means that the object to be identified is a custom object, that is, the matching result is successful, but the description of the custom object is inaccurate. Therefore, feedback from the user terminal on the text information of the object to be identified can be received to accurately adjust the text description of the object to be identified. When the user does not confirm the object image of the object to be identified and the text information of the object to be identified, it means that the object to be identified is not a custom object, that is, the matching result is a failure.

[0114] In one embodiment, Figure 8 As shown, an object recognition method is provided, which is applied to Figure 1 Taking the processor 104 in the example as an example, the following steps may be included:

[0115] S802: When a modification operation of the text information of the object to be confirmed and a confirmation operation of the object image to be identified are received from the user terminal, determining that the matching result is a successful match.

[0116] S804, obtaining the modified object text information and the modified object text features corresponding to the modified object text information.

[0117] In this embodiment, the modified object text information refers to the information after the user modifies the object text information to be confirmed. By inputting the modified object text information into the language model, the modified object text features can be obtained.

[0118] S806, obtaining modified standard text features according to the modified object text features and the standard text features.

[0119] In this embodiment, the modified standard text feature is the average feature between the modified object text feature and the standard text feature. In the actual processing process of the processor, the modified standard text feature, the modified object text feature and the standard text feature are all features represented by numbers. Therefore, the modified standard text feature can be obtained by performing average processing on the digital content corresponding to the modified object text feature and the digital content corresponding to the standard text feature.

[0120] In summary, based on Figure 8 The method shown can continuously update the standard text features of the custom object through the continuous feedback of the user on the text information of the custom object, thereby improving the recognition accuracy when the object in the video stream image is recognized according to the standard text features of the custom object.

[0121] In combination with the above, in one embodiment, Fig. 9 As shown, an object recognition method is provided, which is applied to Figure 1 Taking the processor 104 in the example as an example, the following steps may be included:

[0122] S902, receiving images of custom objects at different perspectives and text information of images at different perspectives uploaded by a user terminal, inputting the images of custom objects at different perspectives into a visual model to obtain standard image features of the custom objects, and inputting the text information of images at different perspectives into a language model to obtain standard text features corresponding to the standard image features.

[0123] S904, receiving a video image uploaded by a user terminal and a video description corresponding to the video image, inputting the video image into a visual model to obtain standard image features of a custom object, and inputting the video description into a language model to obtain standard text features corresponding to the standard image features.

[0124] S906, obtain the video stream image, input the video stream image into the visual model, obtain the object image features of the object to be identified, input the object image features into the image understanding model to obtain the text information corresponding to the video stream image, input the text information corresponding to the video stream image into the language model, and obtain the object text features of the object to be identified.

[0125] When comparing the object image feature of the object to be identified with the standard image feature of the custom object, and comparing the object text feature of the object to be identified with the standard text feature of the custom object, the processor executes different processing logics according to different matching results. Specifically, according to different matching results, the processor may execute S908, or the processor may execute S910.

[0126] S908, when the similarity between the object image feature and the standard image feature is greater than or equal to the image similarity threshold, and the similarity between the object text feature and the standard text feature is greater than or equal to the text similarity threshold, determine that the matching result is a successful match, and mark the custom object in the video stream image.

[0127] S910, when the similarity between the object image feature and the standard image feature is greater than or equal to the image similarity threshold, and the similarity between the object text feature and the standard text feature is less than the text similarity threshold, output the object image to be identified and the object text information to be confirmed to the user terminal.

[0128] The processor may execute different processing logics according to the confirmation status of the object image to be identified and the text information of the object to be confirmed by the user terminal. Specifically, the processor may execute S912, or the processor may execute S914-S918.

[0129] S912: When receiving a confirmation operation of the user terminal on the object image to be identified and the text information of the object to be confirmed, determining that the matching result is a successful match, and marking the custom object in the video stream image.

[0130] S914, when receiving a modification operation of the user terminal on the text information of the object to be confirmed and a confirmation operation on the object image to be identified, determining that the matching result is a successful match, and marking the custom object in the video stream image.

[0131] S916, obtaining the modified object text information and the modified object text features corresponding to the modified object text information.

[0132] S918, obtaining modified standard text features according to the modified object text features and the standard text features.

[0133] Among them, the specific content of S902-S918 can be adapted to the description of the above content and will not be repeated here.

[0134] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0135] Based on the same inventive concept, the embodiment of the present application also provides an object recognition device for implementing the object recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more object recognition device embodiments provided below can refer to the limitations on the object recognition method above, and will not be repeated here.

[0136] In one embodiment, Fig.10 As shown, an object recognition device is provided, including: an acquisition module 1002, a processing module 1004 and a marking module 1006, wherein: the acquisition module 1002 is used to acquire a video stream image and extract object features of an object to be recognized in the video stream image; the processing module 1004 is used to match the object features with standard features of a custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of object information of the custom object; and the marking module 1006 is used to mark the custom object in the video stream image if the matching result is a successful match.

[0137] In one embodiment, the object recognition device also includes a construction module, which is used to: perform three-dimensional reconstruction of the custom object based on the object information to obtain a three-dimensional image, and output the three-dimensional image to the user terminal; when receiving a confirmation operation of the user terminal on the three-dimensional image, obtain standard features of the custom object based on the object information.

[0138] In one embodiment, the construction module is further used to: receive a modification operation of the user terminal on the object information, obtain the modified object information and a modified three-dimensional image corresponding to the modified object information; and when receiving a confirmation operation of the user terminal on the modified three-dimensional image, obtain standard features according to the modified object information.

[0139] In one embodiment, the acquisition module is also used to perform at least one of the following: the first: receiving images of custom objects at different perspectives and text information of images at different perspectives uploaded by a user terminal; the object information of the custom object includes images of the custom object at different perspectives and text information corresponding to the images at different perspectives; the second: receiving video images uploaded by a user terminal and video descriptions corresponding to the video images; the object information of the custom object includes video images of the custom object and video descriptions corresponding to the video images.

[0140] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; the processing module is also used to: when the similarity between the object image features and the standard image features is greater than or equal to the image similarity threshold, and the similarity between the object text features and the standard text features is greater than or equal to the text similarity threshold, determine that the matching result is a successful match.

[0141] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; the processing module is also used to: when the similarity between the object image features and the standard image features is greater than or equal to the image similarity threshold, and the similarity between the object text features and the standard text features is less than the text similarity threshold, output the object image of the object to be identified and the object text information to be confirmed to the user terminal; when receiving the confirmation operation of the user terminal on the object image of the object to be identified and the object text information to be confirmed, determine that the matching result is a successful match.

[0142] In one embodiment, the processing module is also used to: when receiving a modification operation of the user terminal on the text information of the object to be confirmed and a confirmation operation on the object image of the object to be identified, determine that the matching result is a successful match; the object identification device also includes an updating module, and the updating module is used to: obtain the modified object text information and the modified object text features corresponding to the modified object text information; obtain the modified standard text features according to the modified object text features and the standard text features.

[0143] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; the processing module is also used to: when the similarity between the object image features and the standard image features is less than the image similarity threshold, and the similarity between the object text features and the standard text features is less than the text similarity threshold, determine that the matching result is a matching failure.

[0144] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; the processing module is also used to: when the similarity between the object image features and the standard image features is less than the image similarity threshold, and the similarity between the object text features and the standard text features is greater than or equal to the text similarity threshold, determine that the matching result is a matching failure.

[0145] Each module in the above object recognition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in the object recognition system in the form of hardware, or can be stored in a memory in the object recognition system in the form of software, so that the processor can call and execute operations corresponding to each module.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: acquiring a video stream image, extracting object features of an object to be identified in the video stream image; matching the object features with standard features of a custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of object information of the custom object; and if the matching result is a successful match, marking the custom object in the video stream image.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: three-dimensionally reconstructing the custom object according to the object information to obtain a three-dimensional image, and outputting the three-dimensional image to a user terminal; when a confirmation operation of the user terminal on the three-dimensional image is received, standard features of the custom object are obtained according to the object information.

[0148] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: receiving a modification operation of the user terminal on the object information, obtaining the modified object information and a modified three-dimensional image corresponding to the modified object information; when receiving a confirmation operation of the user terminal on the modified three-dimensional image, obtaining standard features according to the modified object information.

[0149] In one embodiment, when the computer program is executed by the processor, at least one of the following steps is also implemented: the first step: receiving images of custom objects at different perspectives and text information of images at different perspectives uploaded by a user terminal; the object information of the custom object includes images of the custom object at different perspectives and text information corresponding to the images at different perspectives; the second step: receiving a video image uploaded by a user terminal and a video description corresponding to the video image; the object information of the custom object includes a video image of the custom object and a video description corresponding to the video image.

[0150] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is greater than or equal to the image similarity threshold, and the similarity between the object text features and the standard text features is greater than or equal to the text similarity threshold, the matching result is determined to be a successful match.

[0151] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is greater than or equal to the image similarity threshold, and the similarity between the object text features and the standard text features is less than the text similarity threshold, the object image of the object to be identified and the object text information to be confirmed are output to the user terminal; when a confirmation operation of the user terminal on the object image of the object to be identified and the object text information to be confirmed is received, the matching result is determined to be a successful match.

[0152] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when receiving a modification operation of the user terminal on the text information of the object to be confirmed and a confirmation operation on the object image of the object to be identified, determining that the matching result is a successful match; obtaining the modified object text information and the modified object text features corresponding to the modified object text information; and obtaining the modified standard text features based on the modified object text features and the standard text features.

[0153] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is less than the image similarity threshold, and the similarity between the object text features and the standard text features is less than the text similarity threshold, the matching result is determined to be a matching failure.

[0154] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is less than the image similarity threshold, and the similarity between the object text features and the standard text features is greater than or equal to the text similarity threshold, the matching result is determined to be a matching failure.

[0155] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the following steps: acquiring a video stream image, extracting object features of an object to be identified in the video stream image; matching the object features with standard features of a custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of object information of the custom object; and if the matching result is a successful match, marking the custom object in the video stream image.

[0156] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: three-dimensionally reconstructing the custom object according to the object information to obtain a three-dimensional image, and outputting the three-dimensional image to a user terminal; when a confirmation operation of the user terminal on the three-dimensional image is received, standard features of the custom object are obtained according to the object information.

[0157] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: receiving a modification operation of the user terminal on the object information, obtaining the modified object information and a modified three-dimensional image corresponding to the modified object information; when receiving a confirmation operation of the user terminal on the modified three-dimensional image, obtaining standard features according to the modified object information.

[0158] In one embodiment, when the computer program is executed by the processor, at least one of the following steps is also implemented: the first step: receiving images of custom objects at different perspectives and text information of images at different perspectives uploaded by a user terminal; the object information of the custom object includes images of the custom object at different perspectives and text information corresponding to the images at different perspectives; the second step: receiving a video image uploaded by a user terminal and a video description corresponding to the video image; the object information of the custom object includes a video image of the custom object and a video description corresponding to the video image.

[0159] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is greater than or equal to the image similarity threshold, and the similarity between the object text features and the standard text features is greater than or equal to the text similarity threshold, the matching result is determined to be a successful match.

[0160] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is greater than or equal to the image similarity threshold, and the similarity between the object text features and the standard text features is less than the text similarity threshold, the object image of the object to be identified and the object text information to be confirmed are output to the user terminal; when a confirmation operation of the user terminal on the object image of the object to be identified and the object text information to be confirmed is received, the matching result is determined to be a successful match.

[0161] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when receiving a modification operation of the user terminal on the text information of the object to be confirmed and a confirmation operation on the object image of the object to be identified, determining that the matching result is a successful match; obtaining the modified object text information and the modified object text features corresponding to the modified object text information; and obtaining the modified standard text features based on the modified object text features and the standard text features.

[0162] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is less than the image similarity threshold, and the similarity between the object text features and the standard text features is less than the text similarity threshold, the matching result is determined to be a matching failure.

[0163] In one embodiment, the object features include object image features of the object to be identified and object text features corresponding to the object image features, and the standard features include standard image features and standard text features corresponding to the standard image features; when the computer program is executed by the processor, the following steps are also implemented: when the similarity between the object image features and the standard image features is less than the image similarity threshold, and the similarity between the object text features and the standard text features is greater than or equal to the text similarity threshold, the matching result is determined to be a matching failure.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0165] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An object recognition method, characterized in that: The method comprises: Acquire a video stream image, and extract object features of an object to be identified in the video stream image; Matching the object features with the standard features of the custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of the object information of the custom object; If the matching result is a successful match, the custom object is marked in the video stream image.

2. The method according to claim 1, characterized in that The construction method of the standard features includes: Reconstructing the custom object in three dimensions according to the object information to obtain a three-dimensional image, and outputting the three-dimensional image to a user terminal; When a confirmation operation of the user terminal on the three-dimensional image is received, standard features of the custom object are obtained according to the object information.

3. The method according to claim 2, characterized in that The method further comprises: receiving a modification operation of the user terminal on the object information, and obtaining the modified object information and a modified three-dimensional image corresponding to the modified object information; When a confirmation operation of the user terminal on the modified three-dimensional image is received, the standard feature is obtained according to the modified object information.

4. The method according to claim 1, characterized in that: The method for obtaining the object information of the custom object includes at least one of the following: The first one: Receiving images of the custom object at different viewing angles and text information of the images at different viewing angles uploaded by a user terminal; the object information of the custom object includes the images of the custom object at different viewing angles and text information corresponding to the images at different viewing angles; Second type: The video image uploaded by the user terminal and the video description corresponding to the video image are received; the object information of the custom object includes the video image of the custom object and the video description corresponding to the video image.

5. The method according to claim 1, characterized in that The object features include the object image features of the object to be identified and the object text features corresponding to the object image features, and the standard features include the standard image features and the standard text features corresponding to the standard image features; The matching of the object features with the standard features of the custom object to obtain a matching result includes: When the similarity between the object image feature and the standard image feature is greater than or equal to an image similarity threshold, and the similarity between the object text feature and the standard text feature is greater than or equal to a text similarity threshold, the matching result is determined to be a successful match.

6. The method according to claim 1 or 5, characterized in that: The object features include the object image features of the object to be identified and the object text features corresponding to the object image features, and the standard features include the standard image features and the standard text features corresponding to the standard image features; the method further includes: When the similarity between the object image feature and the standard image feature is greater than or equal to the image similarity threshold, and the similarity between the object text feature and the standard text feature is less than the text similarity threshold, outputting the object image of the object to be identified and the text information of the object to be confirmed to the user terminal; When a confirmation operation of the user terminal on the object image of the object to be identified and the text information of the object to be confirmed is received, it is determined that the matching result is a successful match.

7. The method according to claim 6, characterized in that The method further comprises: When receiving a modification operation of the user terminal on the text information of the object to be confirmed and a confirmation operation on the object image of the object to be identified, determining that the matching result is a successful match; Obtaining modified object text information and modified object text features corresponding to the modified object text information; According to the modified object text feature and the standard text feature, a modified standard text feature is obtained.

8. The method according to claim 1 or 5, characterized in that: The object features include the object image features of the object to be identified and the object text features corresponding to the object image features, and the standard features include the standard image features and the standard text features corresponding to the standard image features; the method further includes: When the similarity between the object image feature and the standard image feature is less than an image similarity threshold, and the similarity between the object text feature and the standard text feature is less than a text similarity threshold, the matching result is determined to be a matching failure.

9. The method according to claim 1 or 5, characterized in that: The object features include the object image features of the object to be identified and the object text features corresponding to the object image features, and the standard features include the standard image features and the standard text features corresponding to the standard image features; the method further includes: When the similarity between the object image feature and the standard image feature is less than an image similarity threshold, and the similarity between the object text feature and the standard text feature is greater than or equal to a text similarity threshold, the matching result is determined to be a matching failure.

10. An object recognition system, characterized in that: The system includes a camera device and a processor; The camera device is used to obtain video stream images and transmit the video stream images to the processor; The processor is used to extract object features of the object to be identified in the video stream image; match the object features with standard features of the custom object to obtain a matching result; wherein the standard features are determined based on three-dimensional reconstruction of the object information of the custom object; if the matching result is a successful match, mark the custom object in the video stream image.

Images