Image recognition method and device, electronic equipment and storage medium

Abstract

This disclosure provides an image recognition method, apparatus, electronic device, and storage medium. The method involves acquiring a first instruction, which indicates target features of an object; acquiring an environmental image; and detecting the environmental image based on the first instruction to obtain a first recognition object and a second recognition object. The first recognition object is an image element representing target features, and the second recognition object is a text element representing target features. Based on the first and second recognition objects, a target object with target features is identified. By performing target recognition from both image and text elements, and using text elements as reference information for image elements, more accurate target recognition is achieved, thereby accurately identifying target objects in the environmental image, improving recognition accuracy, and solving problems such as low recognition accuracy and slow recognition speed.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, and in particular to an image recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] Image recognition technology refers to the technology of using computing devices to analyze and understand images in order to identify target objects and entities within them. It is widely used in many fields such as security detection and autonomous navigation. For example, in smart terminals designed for visually impaired individuals, image recognition technology is used to capture environmental images, perform image recognition, and convert the recognition results into voice prompts for broadcast, thereby providing target guidance and walking navigation for visually impaired users.

[0003] However, existing image recognition technologies still suffer from problems such as low recognition accuracy and slow recognition speed in complex scenarios. Summary of the Invention

[0004] This disclosure provides an image recognition method, apparatus, electronic device, and storage medium to overcome the problems of low recognition accuracy and slow recognition speed when performing image recognition in complex scenes.

[0005] In a first aspect, embodiments of this disclosure provide an image recognition method, including:

[0006] Obtain a first instruction, which is used to indicate the target features of an object; acquire an environmental image, and detect the environmental image based on the first instruction to obtain a first identification object and a second identification object, wherein the first identification object is an image element representing the target features, and the second identification object is a text element representing the target features; identify a target object having the target features based on the first identification object and the second identification object.

[0007] In a second aspect, embodiments of this disclosure provide an image recognition device, comprising:

[0008] A transceiver module is used to acquire a first instruction, which is used to indicate the target features of an object.

[0009] The processing module is used to acquire an environmental image and detect the environmental image based on the first instruction to obtain a first recognition object and a second recognition object, wherein the first recognition object is an image element characterizing the target feature and the second recognition object is a text element characterizing the target feature;

[0010] The identification module is used to identify a target object having the target features based on the first identification object and the second identification object.

[0011] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0012] A processor, and a memory communicatively connected to the processor;

[0013] The memory stores computer-executed instructions;

[0014] The processor executes computer execution instructions stored in the memory to implement the image recognition method as described in the first aspect and various possible designs of the first aspect.

[0015] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image recognition method described in the first aspect and various possible designs of the first aspect.

[0016] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the image recognition method as described in the first aspect and various possible designs of the first aspect.

[0017] The image recognition method, apparatus, electronic device, and storage medium provided in this embodiment acquire a first instruction, which indicates the target features of an object; acquire an environmental image, and detect the environmental image based on the first instruction to obtain a first recognition object and a second recognition object, wherein the first recognition object is an image element representing the target features, and the second recognition object is a text element representing the target features; and identify a target object having the target features based on the first recognition object and the second recognition object. By detecting the environmental image through a first instruction input by the user to indicate the target features, a first recognition object representing an image element and a second recognition object representing a text element are obtained respectively. Then, using the first and second recognition objects, target recognition is performed from two dimensions: image elements and text elements. Using text elements as reference information for image elements enables more accurate target recognition, thereby accurately identifying target objects in the environmental image, improving recognition accuracy, and solving problems such as low recognition accuracy and slow recognition speed. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is an application scenario diagram of the image recognition method provided in the embodiments of this disclosure;

[0020] Figure 2 Flowchart of the image recognition method provided in the embodiments of this disclosure Figure 1 ;

[0021] Figure 3 for Figure 2 A flowchart of one possible implementation of step S103 in the illustrated embodiment;

[0022] Figure 4 This is a schematic diagram illustrating a process for detecting environmental images based on instruction keywords, provided in an embodiment of this disclosure.

[0023] Figure 5 for Figure 2 A flowchart of another possible implementation of step S103 in the illustrated embodiment;

[0024] Figure 6 This is a schematic diagram illustrating the identification of a second identification object according to an embodiment of the present disclosure;

[0025] Figure 7 for Figure 2 A flowchart illustrating the specific implementation steps of step S104 in the illustrated embodiment;

[0026] Figure 8 Flowchart of the image recognition method provided in the embodiments of this disclosure Figure 2 ;

[0027] Figure 9 for Figure 8 A flowchart illustrating the specific implementation steps of step S209 in the illustrated embodiment;

[0028] Figure 10 This is a schematic diagram illustrating a process for determining a target object, provided by an embodiment of the present disclosure.

[0029] Figure 11 This is a structural block diagram of an image recognition device provided in an embodiment of the present disclosure;

[0030] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0031] Figure 13 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0034] The application scenarios of the embodiments of this disclosure are explained below:

[0035] The image recognition method provided in this disclosure can be applied to scenarios such as image information retrieval, target tracking, and automatic navigation. More specifically, it can be applied to interactive navigation scenarios for visually impaired individuals. The method provided in this disclosure can be applied to terminal devices, such as wearable devices like smart glasses and smart headphones, or other electronic devices with specific computing capabilities. The terminal device is equipped with an image acquisition unit, such as a high-definition camera. After the user wears such a terminal device, the device acquires environmental images of the surrounding environment in real time through the image acquisition unit. When the terminal device receives a user's voice command, it can recognize targets in the environmental image based on the user's command. Once a target object is identified in the environmental image, its location is announced to the user via voice, thus providing travel guidance and walking navigation for visually impaired users. More specifically... Figure 1 This is an application scenario diagram of the image recognition method provided in the embodiments of this disclosure, for example... Figure 1 As shown, the user-wearable terminal device ( Figure 1 After receiving the user's voice command "Search for a taxi with license plate number A12345," the smart glasses will use a high-definition camera to perform image recognition on vehicles driving on the roadside. When a taxi with license plate number A12345 is detected, it will issue a voice prompt to the user, "The target vehicle is 3 meters in front of you." This enables automatic guidance for visually impaired users in the scenario of taking a taxi.

[0036] In related technologies, image recognition technology is typically implemented by pre-training image recognition models to extract and classify image features, thereby identifying target objects with specific image features. For example, based on the image features of "people," "vehicles," and "zebra crossings," detection and matching are performed in the image. When the image feature of "people" is detected in the image, the image element corresponding to that feature is identified as "people," thus achieving the detection of "people" in the image. However, the aforementioned image feature-based target recognition schemes are limited by the size and training cost of image recognition models, and can usually only identify general object categories, failing to achieve more precise target recognition. For example, using a pre-trained image recognition module in related technologies can identify "people" and "buses" in an image, but cannot identify similar-looking "Route 1 bus" and "Route 2 bus." This leads to problems such as low recognition accuracy in more complex application scenarios, such as guiding visually impaired people, and slow recognition speed and increased time consumption due to repeated execution of the recognition algorithm because high-scoring target objects cannot be obtained.

[0037] This disclosure provides an image recognition method to solve the above-mentioned problems.

[0038] refer to Figure 2 , Figure 2 Flowchart of the image recognition method provided in the embodiments of this disclosure Figure 1 The method of this embodiment can be applied in a terminal device. This image recognition method includes:

[0039] Step S101: Obtain the first instruction, which is used to indicate the target features of the object.

[0040] For example, the execution subject in this embodiment is a terminal device, and more specifically, a wearable device such as smart glasses or smart headphones. The first instruction is obtained in a corresponding manner based on different user interaction interfaces of the terminal device. In one possible implementation, the terminal device has a microphone unit for receiving sound signals. When the user needs to detect a target object in the current environment, the user emits voice representing the characteristics of the object. The terminal device converts the voice information received by the microphone unit to obtain the first instruction. In another possible implementation, the terminal device has a user operation panel. The user inputs operation information to the terminal device by tapping or other operations on the operation panel. The terminal device converts the operation information and generates the corresponding first instruction.

[0041] Furthermore, the first instruction is information used to indicate the target features of an object. For example, the content represented in the first instruction may be "detect bus route 1" or "detect taxi with license plate number A12345". Here, "bus route 1" and "taxi with license plate number A12345" are both ways of expressing target features. Based on the above target features, one or a class of objects with the above target features can be identified, that is, the target object.

[0042] Step S102: Acquire environmental images.

[0043] For example, the terminal device is equipped with an image acquisition unit, such as a high-definition camera. By acquiring images through the high-definition camera, the image of the surrounding area of ​​the current location can be obtained, i.e., the environmental image. The image acquisition unit of the terminal device has a certain image acquisition range, i.e., a certain field of view. Therefore, the environmental image can be a single frame image obtained by the terminal device through a single shot taken by the image acquisition unit, or it can be a multi-frame fusion image with a larger field of view generated by merging multiple single frame images taken by the terminal device through multiple shots taken by the image acquisition unit. It can be set as needed. The specific implementation method is the prior art, which will not be described in detail here.

[0044] In another possible implementation, the terminal device can receive image data sent by other electronic devices such as servers, and the image data includes an environmental image. For example, after the terminal device performs filtering and processing steps on the image data, it can obtain the environmental image from the image data. This can be configured as needed, and will not be elaborated here.

[0045] Step S103: Detect the environmental image based on the first instruction to obtain a first identification object and a second identification object, wherein the first identification object is an image element representing the target feature and the second identification object is a text element representing the target feature.

[0046] For example, after acquiring the environmental image, the terminal device detects the environmental image from two different dimensions based on the target features described in the first instruction, and obtains the corresponding first and second recognition objects respectively. Specifically, on the one hand, the terminal device detects the environmental image from the image feature dimension using image feature recognition technology. When the captured environmental image contains image features corresponding to the target features described in the first instruction, the corresponding image element, i.e., the first recognition object, can be detected from the environmental image. On the other hand, the terminal device detects the environmental image from the text feature dimension using optical character recognition (OCR) technology. When the captured environmental image contains text features corresponding to the target features described in the first instruction, the corresponding text element can be detected from the environmental image.

[0047] For example, such as Figure 3 As shown, one possible implementation of step S103 includes:

[0048] Step S1031: Obtain at least one instruction keyword according to the first instruction.

[0049] Step S1032: Based on at least one instruction keyword, perform image recognition on the environmental image to obtain the corresponding first recognition object.

[0050] Step S1033: Based on at least one instruction keyword, perform text recognition on the environmental image to obtain the corresponding second recognition object.

[0051] For example, instruction keywords are characters or words that can be used to characterize part or all of the meaning of the target features corresponding to the first instruction. Specifically, they are, for example, Chinese characters, phrases, numbers, English words, and combinations thereof. Instruction keywords typically have complete semantics; that is, instruction keywords are based on semantic segmentation. Specifically, for example, if the first instruction characterizes the content of "detecting a taxi with license plate number A12345", then based on the semantics of the first instruction, the corresponding instruction keywords would include, for example, "license plate number A12345" and "taxi".

[0052] Subsequently, in one possible implementation, steps S1032 and S1033 are executed independently, respectively, to perform image recognition and text recognition on the environmental image based on the aforementioned instruction keywords, thereby obtaining a first recognition object representing image elements and a second recognition object representing text elements. Figure 4 This is a schematic diagram illustrating a process for detecting environmental images based on instruction keywords, as provided in an embodiment of this disclosure. Figure 4As shown, the environmental image depicts a car driving on a road. First, the terminal device parses the first instruction, obtaining the instruction keywords "license plate A12345" and "taxi" corresponding to the first instruction. On one hand, based on instruction keyword A "taxi", image recognition is performed on the environmental image. Specifically, for example, based on a preset image recognition model, feature extraction and classification prediction are performed on the environmental image. The prediction results are then compared with "taxi" to identify the vehicle in the environmental image and determine it as the first recognition object. On the other hand, based on instruction keyword B "license plate number A12345", text recognition is performed on the environmental image. Specifically, for example, based on an OCR model, all characters in the environmental image are recognized. The recognized characters are then compared with "license plate number A12345" to identify the license plate number "A12345" of the vehicle in the environmental image and determine it as the second recognition object.

[0053] Of course, it is understandable that during the above process, when the environmental image does not contain target features, there may be a situation where the first identification object and / or the second identification object cannot be obtained. When this situation occurs, the process can return to step S102 to re-acquire the environmental image, thereby achieving continuous detection of target objects in the environment; or, the first identification object and / or the second identification object can be set to a specific value (e.g., 0) or empty (NULL), and after obtaining an abnormal target object in a subsequent step (i.e., the target object is not successfully identified), the process can return to step S102 to re-acquire the environmental image, thereby achieving continuous detection of target objects in the environment.

[0054] In another possible implementation, the first and second identification objects are obtained in a specific order, and the second identification object is obtained by using the first identification object as input. Specifically, as follows: Figure 5 As shown, another possible implementation of step S103 includes:

[0055] Step S1034: Perform image detection on the environmental image based on the first instruction to obtain the first recognition object.

[0056] Step S1035: Based on the first instruction, perform text detection on the first recognition object to obtain the target text elements within a first distance from the first recognition object.

[0057] Step S1036: Obtain the second recognition object based on the target text element.

[0058] For example, based on the image features represented by the first instruction, image detection is performed on the environmental image to obtain one or more recognition objects, such as multiple vehicles driving on the road. Then, based on the text features represented by the first instruction, text detection is performed on the first recognition object to obtain target text located inside or near the first recognition object, such as license plate number located within the vehicle outline, and it is identified as the second recognition object. Figure 6 This is a schematic diagram illustrating the identification of a second identification object provided in an embodiment of the present disclosure, such as... Figure 6 As shown, firstly, based on the information Info_1 representing image features in the first instruction, all first recognition objects "vehicles" in the environmental image are identified; then, based on the information Info_2 representing text features in the first instruction, OCR text recognition is performed from the recognition box corresponding to "vehicles" in the environmental image to obtain the target text element "AB12345" that matches the information Info_2, and it is determined as the second recognition element.

[0059] In this embodiment, by first determining the first recognition object and then using the first recognition object as input, the target text element in the environmental image within a first distance from the first recognition object is detected, thereby narrowing the range of text recognition required in the environmental image, thus improving the speed and accuracy of obtaining the target text element, and realizing the fast and accurate recognition of the second recognition object.

[0060] Step S104: Identify the target object with target features based on the first identification object and the second identification object.

[0061] For example, after obtaining the first and second identification objects through the above steps, the positional relationship between the first and second identification objects is used to determine whether the first identification object is the target object, thereby achieving the identification of the target object. Specifically, as shown... Figure 7 As shown, the specific implementation steps of step S204 include:

[0062] Step S1041: Obtain the first spatial coordinates corresponding to the first recognition object and the second spatial coordinates corresponding to the second recognition object.

[0063] Step S1042: Based on the first spatial coordinates and the second spatial coordinates, obtain the spatial distance between the first identification object and the second identification object.

[0064] Step S1043: When the spatial distance is less than the distance threshold corresponding to the first identified object, determine the target position of the target object based on the location of the first identified object.

[0065] Specifically, since the first identification object is an image element representing the target feature and the second identification object is a text element representing the target feature, one or more image elements with the same image feature dimension as the target object can be identified based on the first identification object, such as multiple vehicles with similar shapes. The text element representing the target feature, such as the "license plate number" in the above embodiment, can be identified based on the second identification object. Then, based on the distance relationship between the first and second identification objects, the attribution relationship between the first and second identification objects is determined, i.e., the "vehicle" to which the "license plate number" belongs. Specifically, when the distance between the first and second identification objects is greater than a distance threshold, the text element corresponding to the second identification object usually does not belong to the image element corresponding to the first identification object, i.e., the "license plate number" and the "vehicle" do not correspond to the same target object. Conversely, when the distance between the first and second identification objects is less than the distance threshold, the text element corresponding to the second identification object usually belongs to the image element corresponding to the first identification object, i.e., the "license plate number" and the "vehicle" correspond to the same target object. The distance threshold can be a fixed value or a dynamic value determined based on the characteristics of the first identified object, such as the spatial location, object category, and outline size of the first identified object.

[0066] Furthermore, based on the first identification object and the second identification object, if the positional relationship between the two meets the preset requirements, such as the spatial distance between the first identification object and the second identification object being less than a distance threshold, then it is determined that the first identification object and the second identification object correspond to the same target object. Consequently, the first identification object representing the image element of the target object is the target object; the location of the first identification object is the target location of the target object.

[0067] In one possible scenario, an environmental image typically contains multiple sets of first and second identification objects. When using the second identification object (text features) as reference information for the first identification object (image features) to determine the first identification object as the target object, matching the first and second identification objects is necessary. In this embodiment, the positional relationship between the first and second identification objects is used to achieve matching, thereby enabling the second identification object to serve as reference information for the first identification object in determining the target object, thus improving the efficiency and accuracy of target object recognition.

[0068] Of course, another possible implementation is to convert the first and second identification objects into their corresponding semantics and then match them to achieve the above purpose. The specific implementation method is the existing technology and will not be elaborated here.

[0069] In this embodiment, a first instruction is obtained, which indicates the target features of an object; an environmental image is acquired, and the environmental image is detected based on the first instruction to obtain a first identification object and a second identification object. The first identification object is an image element representing the target features, and the second identification object is a text element representing the target features. Based on the first and second identification objects, a target object with the target features is identified. By detecting the environmental image using the first instruction input by the user to indicate the target features, a first identification object representing image elements and a second identification object representing text elements are obtained. Then, using the first and second identification objects, target recognition is performed from both image and text element dimensions. Using text elements as reference information for image elements enables more accurate target recognition, thereby accurately identifying target objects in the environmental image, improving recognition accuracy, and solving problems such as low recognition accuracy and slow recognition speed.

[0070] refer to Figure 8 , Figure 8 Flowchart of the image recognition method provided in the embodiments of this disclosure Figure 2 This embodiment is in Figure 2 Based on the illustrated embodiment, steps S102 and S104 are further refined, and a user interaction step is added. This image recognition method includes:

[0071] Step S201: Obtain the voice command input by the user, which is used to indicate the target features of the object.

[0072] Step S202: Acquire environmental images.

[0073] For example, step S201 is a specific implementation of the terminal device obtaining the first instruction, namely, obtaining the user's voice command through a microphone unit that receives sound signals. The specific implementations of steps S201 and S202 are as follows: Figure 2 The embodiments shown have been described in detail and will not be repeated here.

[0074] Step S203: Perform speech recognition on the voice command to obtain the corresponding command statement.

[0075] Step S204: Decompose the instruction statement according to the first semantic feature corresponding to the instruction statement to obtain at least one instruction keyword in the instruction statement.

[0076] For example, after receiving a voice command, the voice command can be an audio signal, which is then subjected to speech recognition to obtain the text content it expresses, i.e., the command statement. The specific implementation method of performing natural language recognition on the audio signal to obtain the corresponding natural language statement is existing technology and will not be elaborated here.

[0077] Further, exemplarily, after receiving the instruction statement, the characters or words constituting the instruction statement are decomposed, and meaningless characters or words are filtered out to obtain one or more keywords that can characterize the target features, i.e., instruction keywords. The specific meaning of the instruction keywords is as follows: Figure 2 The embodiments shown have already been described and will not be repeated here. For example, if the command statement obtained after speech recognition is "Find me a taxi with license plate number A12345", then the command keywords obtained after decomposing the command statement are, for example, "taxi" and "A12345".

[0078] For example, since user-issued voice commands have a certain degree of subjectivity and arbitrariness, when the generated command statements are relatively complex, the splitting and filtering of command statements need to follow different rules. In this embodiment, the command statements are analyzed based on the first semantic features corresponding to the command statements. The first semantic features corresponding to the command statements are information used to describe the content and context expressed by the command statements. For example, if the command statement is "Find the No. 1 bus to location A", then according to the content and context expressed by the command statement (i.e., the first semantic features), the corresponding command keywords are "go", "No. A", "No. 1", and "bus". When the command statement is "go to No. 1 bus", then according to the content and context expressed by the command statement, the command keywords are "No. 1" and "bus". According to the first semantic features, it can be determined that the word "go" does not represent the actual meaning, that is, it does not represent the target feature. Therefore, the command keywords obtained do not contain "go".

[0079] In this embodiment, the first semantic feature corresponding to the voice command is obtained to achieve accurate analysis of the command statement, thereby improving the accuracy of command keywords and the accuracy of target object search and positioning.

[0080] Optionally, after step S204, the method further includes:

[0081] Step S205: Obtain the second semantic features corresponding to each instruction keyword, and based on the second semantic features, obtain the approximate keywords corresponding to the instruction keywords.

[0082] For example, after obtaining the instruction keyword, to reduce the impact of the randomness of the user's voice command on the search process, near-synonyms or synonyms with similar meanings, i.e., approximate keywords, can be generated based on the second semantic feature of the instruction keyword. The second semantic feature represents the meaning and content of the instruction keyword. For instance, if the instruction keyword is "bus," then based on its second semantic feature, corresponding approximate keywords such as "bus" or "public transport" can be obtained from a preset vocabulary. Subsequently, when searching for text elements in an environmental image, the instruction keyword and its corresponding approximate keywords are used together to improve the matching efficiency and scope of target text elements, thereby improving the accuracy of target object recognition.

[0083] Step S206: Obtain the keyword category of the instruction keyword. The keyword category includes at least a first category or a second category. The second semantic feature of the instruction keyword in the first category is represented by image features, and the second semantic feature of the instruction keyword in the second category is represented by text features.

[0084] For example, the instruction keyword has a corresponding keyword category, wherein the keyword category includes at least a first category or a second category, that is, the instruction keyword can be an instruction keyword of the first category, or the instruction keyword can be an instruction keyword of the second category. Specifically, the second semantic feature of the instruction keyword of the first category is represented by image features, that is, the content of the instruction keyword of the first category can be represented by image features. More specifically, for example, the instruction keyword of the first category can be "car", "pedestrian", "shelf", "door", etc. The second semantic feature of the instruction keyword of the second category is represented by text features, that is, the content of the instruction keyword of the second category can be represented by text features. More specifically, for example, the instruction keyword of the second category can be "Route 2", "A12345", "men's restroom", etc. The content of the instruction keyword and the keyword category have a preset mapping relationship. For example, by recognizing the instruction keyword through a pre-trained recognition model, the keyword category corresponding to the instruction keyword can be obtained.

[0085] In one possible implementation, the keyword category also includes a third category, meaning that the instruction keyword can be of the first category, the second category, or the third category. Specifically, the second semantic features of the instruction keywords in the first category are represented solely by image features; the second semantic features of the instruction keywords in the second category are represented solely by text features; and the second semantic features of the instruction keywords in the third category can be represented by both image features and text features.

[0086] Step S207: Based on at least one instruction keyword of the first category and the corresponding approximate keyword, perform image recognition on the environmental image to obtain the corresponding first recognition object.

[0087] Step S208: Based on at least one second-category instruction keyword and the corresponding approximate keyword, perform text recognition on the environmental image to obtain the corresponding second recognition object.

[0088] For example, after obtaining the keyword category corresponding to the instruction keyword, corresponding steps are performed according to the keyword category to obtain the corresponding first and second recognition objects. Specifically, for example, on the one hand, image elements in the environmental image are segmented and corresponding image features are extracted. Then, the image features corresponding to each image element are mapped to descriptive text, such as "car" or "bus," and compared with the instruction keywords of the first or third category. If they match, the image element is identified as the first recognition object. On the other hand, OCR technology is used to recognize and extract text features in the environmental image to obtain text elements, such as "Route 2," "Men's Restroom," or "A12345," and compared with the instruction keywords of the second or third category. If they match, the text element is identified as the second recognition object.

[0089] In this embodiment, by classifying instruction keywords, distinguishing between first and second types of instruction keywords, and determining the corresponding first and second identification objects based on different instruction types, the accurate detection of image elements and text elements with target features in the environmental image is achieved, thereby improving the accuracy of subsequent target object determination.

[0090] Step S209: Obtain the distance threshold corresponding to the first identified object.

[0091] For example, in one possible implementation, the distance threshold of the first identified object can be a fixed value determined based on the image scene of the environmental image. For example, when the image scene described by the environmental image is an indoor scene, the distance threshold of the first identified object is 0.2 meters; when the image scene described by the environmental image is an outdoor scene, the distance threshold of the first identified object is 1 meter.

[0092] In another possible implementation, the distance threshold of the first identification object is related to the shape and size of the first identification object. The distance threshold for matching the first identification object is determined based on the shape and size of the first identification object, and then the surrounding text elements are determined to belong to the first identification object based on the distance threshold.

[0093] For example, such as Figure 9 As shown, the specific implementation of step S209 includes:

[0094] Step S2091: Obtain the outline size of the first identified object and / or the object category corresponding to the first identified object.

[0095] Step S2092: Determine the distance threshold corresponding to the first identified object based on the contour size.

[0096] In one possible implementation, for example, after identifying the first identification object (image element) in the environmental image, the contour size of the first identification object, such as the diagonal length of the rectangle surrounding the image element, can be obtained through reference objects in the environmental image or camera parameters, which will not be elaborated here. Then, based on the contour size and a preset mapping relationship, a corresponding distance threshold is determined, with the contour size being proportional to the distance threshold. In another possible implementation, the object category corresponding to the first identification object can also be identified, such as "bus," and the corresponding distance threshold can be determined based on the object category alone or in combination with the contour size. For example, when the first identification object is "bus," its contour size is larger, and its corresponding distance threshold is also larger; when the first identification object is "toilet door," its contour size is smaller, and its corresponding distance threshold is also smaller. In this embodiment, the corresponding distance threshold is determined by the contour size and / or object category of the first identification object, thereby increasing the probability of correct matching between the first and second identification objects and improving the accuracy of target object identification.

[0097] Step S210: Determine the target position of the target object based on the relationship between the spatial distance and the distance threshold between the first and second identified objects.

[0098] For example, then, based on a distance threshold, the spatial distance between the first identified object and the second identified object is compared. If the spatial distance between the first identified object and the second identified object is less than the distance threshold, the first identified object is determined to be the target object. Then, based on the position of the first identified object, the target position is determined. After that, the terminal device performs voice broadcast based on the target position to provide prompts to the user. On the other hand, if the spatial distance between the first identified object and the second identified object is greater than the distance threshold, the first identified object is determined not to be the target object. Then, the process returns to step S202 to re-acquire environmental images and achieve continuous detection of the environment where the terminal device is located.

[0099] Among them, the spatial distance between the first recognition object and the second recognition object can be determined by the spatial distance between the center points of the outlines of the first recognition object and the second recognition object. Further, since the environmental image is a two-dimensional image, to calculate the spatial distance between objects in this two-dimensional image, it needs to be mapped to a three-dimensional space for calculation. Therefore, the terminal device needs to obtain a three-dimensional space model of the real environment represented by the environmental image. This three-dimensional space model can be pre-generated or generated based on the environmental image. The specific implementation method is prior art and will not be elaborated here.

[0100] Figure 10 The process schematic diagram of determining a target object provided by an embodiment of the present disclosure is as Figure 10 shown. After recognizing the environmental image, two first recognition objects and two second recognition objects in the environmental image are obtained. Among them, as shown in the figure, the two first recognition objects are T1_1 and T1_2 respectively, and the image elements corresponding to the first recognition objects T1_1 and T1_2 are both buses; the two second recognition objects are T2_1 and T2_2 respectively, and the text elements corresponding to the second recognition objects T2_1 and T2_2 are "Route 2". The above two first recognition objects and two second recognition objects are all recognition results obtained in the environmental image based on the first instruction. After that, the spatial distances D1 to D4 between each of the above first recognition objects and second recognition objects are obtained, and each spatial distance is sequentially compared based on a distance threshold. Among them, D2 is less than the distance threshold, indicating that D2 corresponds to the first recognition object and the second recognition object, belonging to the same object, that is, the text element "Route 2" represented by the second recognition object is a description of the image element "bus" of the first recognition object. Therefore, the first recognition object corresponding to the spatial distance D2 less than the distance threshold is determined as the target object.

[0101] In the steps of this embodiment, by comparing the relationship between the spatial distance between the first recognition object and the second recognition object and the distance threshold, the matching first recognition object and second recognition object are determined, so as to achieve the accurate recognition of the target object, improve the recognition accuracy of the target object, and finally realize functions such as information search, target guidance, and navigation for visually impaired persons.

[0102] Corresponding to the image recognition method in the above embodiment, Figure 11 The structural block diagram of the image recognition device provided by an embodiment of the present disclosure is shown. For the sake of convenience of description, only the parts related to the embodiment of the present disclosure are shown.

[0103] Referring to Figure 11 , the image recognition device 3 includes:

[0104] A transceiver module 31, configured to obtain a first instruction, where the first instruction is used to indicate the target feature of an object;

[0105] The processing module 32 is used to acquire an environmental image and detect the environmental image based on a first instruction to obtain a first recognition object and a second recognition object, wherein the first recognition object is an image element representing the target feature and the second recognition object is a text element representing the target feature;

[0106] The identification module 33 is used to identify a target object with target features based on the first identification object and the second identification object.

[0107] In one embodiment of this disclosure, when the processing module 32 detects an environmental image based on a first instruction to obtain a first recognition object and a second recognition object, it is specifically used to: obtain at least one instruction keyword according to the first instruction; perform image recognition on the environmental image based on the at least one instruction keyword to obtain the corresponding first recognition object; and perform text recognition on the environmental image based on the at least one instruction keyword to obtain the corresponding second recognition object.

[0108] In one embodiment of this disclosure, the first instruction is a voice instruction. When the processing module 32 obtains at least one instruction keyword according to the first instruction, it is specifically used to: perform voice recognition on the voice instruction to obtain the corresponding instruction statement; and decompose the instruction statement according to the first semantic feature corresponding to the instruction statement to obtain at least one instruction keyword in the instruction statement.

[0109] In one embodiment of this disclosure, after the processing module 32 decomposes the instruction statement according to the scene information to obtain at least one instruction keyword in the instruction statement, it is further configured to: obtain a second semantic feature corresponding to the instruction keyword; and based on the second semantic feature, obtain an approximate keyword corresponding to the instruction keyword, wherein the approximate keyword is used to perform text recognition and / or image recognition on the environmental image to obtain a first recognition object and / or a second recognition object.

[0110] In one embodiment of this disclosure, the processing module 32 is further configured to: obtain the keyword category of the instruction keyword, wherein the keyword category includes at least a first category or a second category, wherein the second semantic feature of the instruction keyword of the first category is represented by image features, and the second semantic feature of the instruction keyword of the second category is represented by text features; when the processing module 32 performs image recognition on the environmental image based on at least one instruction keyword to obtain the corresponding first recognition object, it is specifically configured to: perform image recognition on the environmental image based on at least one instruction keyword of the first category to obtain the corresponding first recognition object; when the processing module 32 performs text recognition on the environmental image based on at least one instruction keyword to obtain the corresponding second recognition object, it is specifically configured to: perform text recognition on the environmental image based on at least one instruction keyword of the second category to obtain the corresponding second recognition object.

[0111] In one embodiment of this disclosure, the processing module 32 is specifically used for: performing image detection on an environmental image based on a first instruction to obtain a first recognition object; performing text detection on the first recognition object based on the first instruction to obtain target text elements within a first distance from the first recognition object; and obtaining a second recognition object based on the target text elements.

[0112] In one embodiment of this disclosure, the identification module 33 is specifically used for: obtaining the first spatial coordinates corresponding to the first identification object and the second spatial coordinates corresponding to the second identification object; obtaining the spatial distance between the first identification object and the second identification object based on the first spatial coordinates and the second spatial coordinates; obtaining the distance threshold corresponding to the first identification object; and determining the target position of the target object based on the position of the first identification object when the spatial distance is less than the distance threshold.

[0113] In one embodiment of this disclosure, the identification module 33, when obtaining the distance threshold corresponding to the first identification object, is specifically configured to: obtain the outline size of the first identification object; determine the distance threshold corresponding to the first identification object based on the outline size and / or the object category corresponding to the first identification object; and determine the distance threshold corresponding to the first identification object based on the outline size and / or the object category.

[0114] The transceiver module 31, processing module 32, and recognition module 33 are connected sequentially. The image recognition device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0115] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 12 As shown, the electronic device 4 includes:

[0116] Processor 41, and memory 42 communicatively connected to processor 41;

[0117] Memory 42 stores instructions executed by the computer;

[0118] The processor 41 executes computer execution instructions stored in the memory 42 to achieve, for example, Figures 2-10 The image recognition method in the illustrated embodiment.

[0119] Optionally, the processor 41 and the memory 42 are connected via a bus 43.

[0120] For relevant instructions, please refer to the corresponding text. Figures 2-10 The relevant descriptions and effects of the steps in the corresponding embodiments are understood, and will not be elaborated on here.

[0121] This disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement this application. Figures 2-10 The image recognition method provided in any of the corresponding embodiments.

[0122] refer to Figure 13 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 13 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0123] like Figure 13 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0124] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 13An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0125] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0126] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0127] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0128] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0129] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0131] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0132] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0133] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] In a first aspect, according to one or more embodiments of the present disclosure, an image recognition method is provided, comprising:

[0135] Obtain a first instruction, which is used to indicate the target features of an object; acquire an environmental image, and detect the environmental image based on the first instruction to obtain a first identification object and a second identification object, wherein the first identification object is an image element representing the target features, and the second identification object is a text element representing the target features; identify a target object having the target features based on the first identification object and the second identification object.

[0136] According to one or more embodiments of this disclosure, the step of detecting the environmental image based on the first instruction to obtain a first identification object and a second identification object includes: obtaining at least one instruction keyword based on the first instruction; performing image recognition on the environmental image based on at least one instruction keyword to obtain a corresponding first identification object; and performing text recognition on the environmental image based on at least one instruction keyword to obtain a corresponding second identification object.

[0137] According to one or more embodiments of this disclosure, the first instruction is a voice instruction, and obtaining at least one instruction keyword based on the first instruction includes: performing voice recognition on the voice instruction to obtain a corresponding instruction statement; and decomposing the instruction statement according to a first semantic feature corresponding to the instruction statement to obtain at least one instruction keyword in the instruction statement.

[0138] According to one or more embodiments of this disclosure, after decomposing the instruction statement according to scene information to obtain at least one instruction keyword in the instruction statement, the method further includes: obtaining a second semantic feature corresponding to the instruction keyword; and obtaining an approximate keyword corresponding to the instruction keyword based on the second semantic feature, wherein the approximate keyword is used to perform text recognition and / or image recognition on the environmental image to obtain the first recognition object and / or the second recognition object.

[0139] According to one or more embodiments of this disclosure, the method further includes: obtaining the keyword category of the instruction keyword, the keyword category including at least a first category or a second category, wherein the second semantic feature of the instruction keyword of the first category is represented by image features, and the second semantic feature of the instruction keyword of the second category is represented by text features; the step of performing image recognition on the environmental image based on at least one instruction keyword to obtain a corresponding first recognition object includes: performing image recognition on the environmental image based on at least one instruction keyword of the first category to obtain a corresponding first recognition object; the step of performing text recognition on the environmental image based on at least one instruction keyword to obtain a corresponding second recognition object includes: performing text recognition on the environmental image based on at least one instruction keyword of the second category to obtain a corresponding second recognition object.

[0140] According to one or more embodiments of this disclosure, the step of detecting the environmental image based on the first instruction to obtain a first identification object and a second identification object includes: performing image detection on the environmental image based on the first instruction to obtain a first identification object; performing text detection on the first identification object based on the first instruction to obtain target text elements within a first distance from the first identification object; and obtaining the second identification object based on the target text elements.

[0141] According to one or more embodiments of this disclosure, identifying a target object having the target feature based on the first identification object and the second identification object includes: obtaining a first spatial coordinate corresponding to the first identification object and a second spatial coordinate corresponding to the second identification object; obtaining a spatial distance between the first identification object and the second identification object based on the first spatial coordinate and the second spatial coordinate; obtaining a distance threshold corresponding to the first identification object; and determining the target position of the target object based on the position of the first identification object when the spatial distance is less than the distance threshold corresponding to the first identification object.

[0142] According to one or more embodiments of this disclosure, obtaining a distance threshold corresponding to the first identified object includes: obtaining the outline size of the first identified object and / or the object category corresponding to the first identified object; and determining the distance threshold corresponding to the first identified object based on the outline size and / or the object category.

[0143] Secondly, according to one or more embodiments of this disclosure, an image recognition device is provided, comprising:

[0144] A transceiver module is used to acquire a first instruction, which is used to indicate the target features of an object.

[0145] The processing module is used to acquire an environmental image and detect the environmental image based on the first instruction to obtain a first recognition object and a second recognition object, wherein the first recognition object is an image element characterizing the target feature and the second recognition object is a text element characterizing the target feature;

[0146] The identification module is used to identify a target object having the target features based on the first identification object and the second identification object.

[0147] According to one or more embodiments of this disclosure, when the processing module detects the environmental image based on the first instruction to obtain a first recognition object and a second recognition object, it is specifically configured to: obtain at least one instruction keyword based on the first instruction; perform image recognition on the environmental image based on at least one instruction keyword to obtain a corresponding first recognition object; and perform text recognition on the environmental image based on at least one instruction keyword to obtain a corresponding second recognition object.

[0148] According to one or more embodiments of this disclosure, the first instruction is a voice instruction, and when the processing module obtains at least one instruction keyword according to the first instruction, it is specifically used to: perform voice recognition on the voice instruction to obtain a corresponding instruction statement; and decompose the instruction statement according to the first semantic feature corresponding to the instruction statement to obtain at least one instruction keyword in the instruction statement.

[0149] According to one or more embodiments of this disclosure, after the processing module decomposes the instruction statement according to the scene information to obtain at least one instruction keyword in the instruction statement, it is further configured to: obtain a second semantic feature corresponding to the instruction keyword; and based on the second semantic feature, obtain an approximate keyword corresponding to the instruction keyword, wherein the approximate keyword is used to perform text recognition and / or image recognition on the environmental image to obtain the first recognition object and / or the second recognition object.

[0150] According to one or more embodiments of this disclosure, the processing module is further configured to: obtain the keyword category of the instruction keyword, the keyword category including at least a first category or a second category, wherein the second semantic feature of the instruction keyword of the first category is represented by image features, and the second semantic feature of the instruction keyword of the second category is represented by text features; when the processing module 32 performs image recognition on the environment image based on at least one instruction keyword to obtain a corresponding first recognition object, it is specifically configured to: perform image recognition on the environment image based on at least one instruction keyword of the first category to obtain a corresponding first recognition object; when the processing module 32 performs text recognition on the environment image based on at least one instruction keyword to obtain a corresponding second recognition object, it is specifically configured to: perform text recognition on the environment image based on at least one instruction keyword of the second category to obtain a corresponding second recognition object.

[0151] According to one or more embodiments of this disclosure, the processing module is specifically configured to: perform image detection on the environmental image based on the first instruction to obtain a first recognition object; perform text detection on the first recognition object based on the first instruction to obtain a target text element within a first distance from the first recognition object; and obtain a second recognition object based on the target text element.

[0152] According to one or more embodiments of this disclosure, the identification module is specifically configured to: obtain the first spatial coordinates corresponding to the first identification object and the second spatial coordinates corresponding to the second identification object; obtain the spatial distance between the first identification object and the second identification object based on the first spatial coordinates and the second spatial coordinates; and determine the target position of the target object based on the position of the first identification object when the spatial distance is less than the distance threshold corresponding to the first identification object.

[0153] According to one or more embodiments of this disclosure, the identification module is further configured to: obtain the outline size of the first identified object and / or the object category corresponding to the first identified object; and determine a distance threshold corresponding to the first identified object based on the outline size and / or the object category.

[0154] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, including: a processor, and a memory communicatively connected to the processor;

[0155] The memory stores computer-executed instructions;

[0156] The processor executes computer execution instructions stored in the memory to implement the image recognition method as described in the first aspect and various possible designs of the first aspect.

[0157] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the image recognition method described in the first aspect and various possible designs of the first aspect is implemented.

[0158] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the image recognition method as described in the first aspect and various possible designs of the first aspect.

[0159] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0160] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0161] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image recognition method, characterized in that, include: Obtain a first instruction, which is used to indicate the target features of the object; An environmental image is acquired, and the environmental image is detected based on the first instruction to obtain a first recognition object and a second recognition object, wherein the first recognition object is an image element characterizing the target feature, and the second recognition object is a text element characterizing the target feature; Based on the first identification object and the second identification object, identify the target object having the target feature; The step of identifying a target object having the target features based on the first identification object and the second identification object includes: Obtain the first spatial coordinates corresponding to the first identified object and the second spatial coordinates corresponding to the second identified object; Based on the first spatial coordinates and the second spatial coordinates, the spatial distance between the first identified object and the second identified object is obtained; Obtain the distance threshold corresponding to the first identified object; wherein, the distance threshold is a dynamic value determined based on the characteristics of the first identified object; When the spatial distance is less than the distance threshold, the target position of the target object is determined based on the location of the first identified object.

2. The method according to claim 1, characterized in that, The step of detecting the environmental image based on the first instruction to obtain a first identification object and a second identification object includes: Based on the first instruction, at least one instruction keyword is obtained; Based on at least one of the instruction keywords, image recognition is performed on the environmental image to obtain the corresponding first recognition object; Based on at least one of the instruction keywords, text recognition is performed on the environmental image to obtain the corresponding second recognition object.

3. The method according to claim 2, characterized in that, The first instruction is a voice instruction, and the step of obtaining at least one instruction keyword based on the first instruction includes: The voice command is subjected to speech recognition to obtain the corresponding command statement; Based on the first semantic feature corresponding to the instruction statement, the instruction statement is decomposed to obtain at least one instruction keyword in the instruction statement.

4. The method according to claim 2, characterized in that, After decomposing the instruction statement according to the scenario information to obtain at least one instruction keyword in the instruction statement, the method further includes: Obtain the second semantic feature corresponding to the instruction keyword; Based on the second semantic feature, an approximate keyword corresponding to the instruction keyword is obtained. The approximate keyword is used to perform text recognition and / or image recognition on the environmental image to obtain the first recognition object and / or the second recognition object.

5. The method according to claim 2, characterized in that, The method further includes: Obtain the keyword category of the instruction keyword, wherein the keyword category includes at least a first category or a second category, wherein the second semantic feature of the instruction keyword in the first category is represented by image features, and the second semantic feature of the instruction keyword in the second category is represented by text features; The step of performing image recognition on the environmental image based on at least one of the instruction keywords to obtain the corresponding first recognition object includes: Based on at least one instruction keyword of the first category, image recognition is performed on the environmental image to obtain the corresponding first recognition object; The step of performing text recognition on the environmental image based on at least one of the instruction keywords to obtain the corresponding second recognition object includes: Based on at least one instruction keyword of the second category, text recognition is performed on the environmental image to obtain the corresponding second recognition object.

6. The method according to claim 1, characterized in that, The step of detecting the environmental image based on the first instruction to obtain a first identification object and a second identification object includes: Based on the first instruction, image detection is performed on the environmental image to obtain a first identification object; Based on the first instruction, text detection is performed on the first recognition object to obtain target text elements within a first distance from the first recognition object; The second recognition object is obtained based on the target text element.

7. The method according to claim 1, characterized in that, The step of obtaining the distance threshold corresponding to the first identified object includes: Obtain the outline dimensions of the first identified object and / or the object category corresponding to the first identified object; Based on the contour size and / or the object category, determine the distance threshold corresponding to the first identified object.

8. An image recognition device, characterized in that, include: A transceiver module is used to acquire a first instruction, which is used to indicate the target features of an object. The processing module is used to acquire an environmental image and detect the environmental image based on the first instruction to obtain a first recognition object and a second recognition object, wherein the first recognition object is an image element characterizing the target feature and the second recognition object is a text element characterizing the target feature; The identification module is used to identify a target object having the target features based on the first identification object and the second identification object; The identification module is specifically used to obtain the first spatial coordinates corresponding to the first identification object and the second spatial coordinates corresponding to the second identification object; to obtain the spatial distance between the first identification object and the second identification object based on the first spatial coordinates and the second spatial coordinates; to obtain the distance threshold corresponding to the first identification object; wherein the distance threshold is a dynamic value determined based on the characteristics of the first identification object; when the spatial distance is less than the distance threshold, to determine the target position of the target object based on the position of the first identification object.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the image recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the image recognition method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the image recognition method according to any one of claims 1 to 7.

Images