Information identification method and device, and computer device

Abstract

The application provides an information recognition method and device and a computer device. When an object recognition is performed on a current frame image of collected to-be-recognized text to determine a second recognition area of a to-be-recognized object indicated by an indicating object, the application further obtains a first recognition area of the indicating object. When it is detected that the second recognition area is blocked by the first recognition area and a preset condition is met, the application obtains at least one historical frame image adjacent to the current frame image as a to-be-recognized image. Through recognition processing on the to-be-recognized object contained in the to-be-recognized image, a high-precision and reliable recognition processing result of the to-be-recognized object is obtained. Compared with the mode of still performing object recognition processing on the current frame image, the application improves the reliability and accuracy of the information recognition result.

Description

Technical Field

[0001] This application relates primarily to the field of communication technology, and more specifically to an information identification method, apparatus, and computer equipment. Background Technology

[0002] With the application and development of artificial intelligence in computer equipment in various fields, the application functions of computer equipment have been greatly enriched, providing great benefits to people's lives, work and study.

[0003] Taking an AI-based finger-reading / finger-translation device as an example, after the device is activated, the content of the document to be recognized can be placed within the camera's field of view. The user can point their finger at the object to be recognized. After the camera captures an image containing the content, it identifies the object through image analysis, reads the object, or translates it into the target object and outputs it to meet application requirements.

[0004] However, during the recognition process, if the object to be recognized is obscured by the user's finger, it is easy for the recognized object to be different from the target object that the user actually wants to recognize, which reduces the recognition accuracy and thus affects the accuracy and reliability of finger reading / translation. Summary of the Invention

[0005] In view of the above, this application provides the following technical solution:

[0006] On the one hand, this application proposes an information identification method, the method comprising:

[0007] Obtain the current frame image of the text to be recognized;

[0008] Perform object recognition on the current frame image to determine a first recognition region of the indicated object and a second recognition region of the object to be recognized indicated by the indicated object in the text to be recognized;

[0009] If the second recognition region is detected to be occluded by the first recognition region and a preset condition is met, the historical frame image of at least one adjacent frame of the current frame image is obtained as the image to be recognized.

[0010] The object to be identified contained in the image to be identified is subjected to identification processing, and the identification processing result is output.

[0011] Optionally, detecting that the second recognition area is occluded by the first recognition area and meets a preset condition includes:

[0012] Obtain the overlapping area between the first identification area and the second identification area;

[0013] The area of ​​the overlapping region is detected to be less than a first area threshold; or,

[0014] The area of ​​the overlapping region is detected to be less than a first area threshold and greater than a second area threshold; or,

[0015] The overlapping region is detected to have a length less than a first length threshold in a specified direction; the specified direction refers to the object reading direction of the text to be identified or the direction perpendicular to the object reading direction; or,

[0016] The length of the overlapping region in a specified direction is detected to be less than a first length threshold and greater than a second length threshold.

[0017] Optionally, detecting that the second recognition area is occluded by the first recognition area and meets a preset condition further includes:

[0018] Obtain the display area in the second identification region that is not obscured by the first identification region;

[0019] The area of ​​the display area is detected to be greater than a third area threshold and less than a fourth area threshold; or,

[0020] The area of ​​the display area is detected to be greater than a third area threshold; or,

[0021] It was detected that the length of the display area in the specified direction is greater than a third length threshold and less than a fourth length threshold; or,

[0022] It was detected that the length of the display area in the specified direction is greater than a third length threshold.

[0023] Optionally, the method further includes:

[0024] If the overlapping region is obtained, and it is detected that the area of ​​the overlapping region is less than the second area threshold, or the length of the overlapping region is less than the second length threshold, then the current frame image is determined to be an image to be identified.

[0025] If the display area is obtained and it is detected that the area of ​​the display area is greater than the fourth area threshold, or the length of the display area is greater than the fourth length threshold, then the current frame image is determined to be an image to be identified.

[0026] Optionally, the method further includes:

[0027] If the second recognition region is detected to be occluded by the first recognition region and does not meet the preset condition, the third recognition region of the first adjacent object located in the indication direction of the object to be recognized, as indicated by the indication object in the current frame image, is determined;

[0028] The first adjacent object in the third identification region is identified, and the identification result obtained by the output step is executed.

[0029] Optionally, detecting that the second recognition area is occluded by the first recognition area and does not meet the preset condition includes:

[0030] If the overlapping region is obtained, it is detected that the area of ​​the overlapping region is greater than the first area threshold, or the length of the overlapping region in the specified direction is greater than the first length threshold;

[0031] If the display area is obtained, it is detected that the area of ​​the display area is less than the third area threshold, or the length of the display area in the specified direction is less than the third length threshold.

[0032] Optionally, the step of performing object recognition on the current frame image to determine a first recognition region indicating an object and a second recognition region of the object to be recognized indicated by the indicating object in the text to be recognized includes:

[0033] Object recognition is performed on the current frame image to determine the first recognition region of the indicated object, the object to be recognized pointed to by the indicator vertex of the indicated object, and the second adjacent object of the object to be recognized in the reading direction of the text to be recognized;

[0034] If the second neighboring object in the current frame image is not occluded by the indicated object, the second recognition region of the object to be recognized is determined based on the fourth recognition region of the second neighboring object; the upper and lower edges of the second recognition region are consistent with those of the fourth recognition region.

[0035] Optionally, the step of performing recognition processing on the object to be recognized contained in the image to be recognized and outputting the obtained recognition processing result includes:

[0036] The indicated object in the image to be identified is tracked and detected, and the object to be identified indicated by the indicated object in the current frame image is determined to be in the fifth recognition region of the image to be identified.

[0037] Semantic recognition is performed on the object to be recognized contained in the fifth recognition region to obtain target speech that meets the recognition processing requirements and is semantically consistent with the object to be recognized; the recognition processing requirements include language category requirements for the output speech;

[0038] Output the target speech.

[0039] Furthermore, this application also proposes an information identification device, the device comprising:

[0040] The image acquisition module is used to acquire the current frame image of the text to be recognized;

[0041] The object recognition module is used to perform object recognition on the current frame image, determine a first recognition region of the indicated object and a second recognition region of the object to be recognized indicated by the indicated object in the text to be recognized;

[0042] The image acquisition module is used to detect that the second recognition region is occluded by the first recognition region and meets a preset condition, and to acquire at least one adjacent historical frame image of the current frame image as the image to be recognized.

[0043] The recognition processing module is used to perform recognition processing on the object to be recognized contained in the image to be recognized, and output the obtained recognition processing result.

[0044] In another aspect, this application also proposes a computer device, said computer device comprising:

[0045] The communication module is used to acquire the current frame image of the text to be recognized captured by the image acquisition device;

[0046] The memory is used to store programs that implement the information recognition method described above;

[0047] A processor is used to load and execute the program stored in the memory to implement the information recognition method described above.

[0048] Therefore, this application provides an information recognition method, apparatus, and computer device. When performing object recognition on the current frame image of the acquired text to be recognized and determining the second recognition region of the object to be recognized indicated by the indicator, this application also obtains the first recognition region of the indicator. If the second recognition region is detected to be occluded by the first recognition region and a preset condition is met, this application obtains at least one adjacent historical frame image of the current frame image as the image to be recognized. By performing recognition processing on the aforementioned object to be recognized contained in the image to be recognized, the recognition processing result of the object to be recognized is obtained with high accuracy and reliability. Compared with the method of still performing object recognition processing on the current frame image, this application improves the reliability and accuracy of the information recognition result. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0050] Figure 1 A schematic diagram of the hardware structure of an optional example of a computer device suitable for the information recognition method proposed in this application;

[0051] Figure 2 A schematic diagram of the hardware structure of another optional example of a computer device suitable for the information recognition method proposed in this application;

[0052] Figure 3 This is a schematic diagram of a system architecture for an optional application scenario of the information recognition method proposed in this application;

[0053] Figure 4 This is a schematic diagram of an optional application scenario for the information recognition method proposed in this application;

[0054] Figure 5 This is a schematic diagram of the recognition result of the current frame image in an optional application scenario of the information recognition method proposed in this application;

[0055] Figure 6 This is a schematic diagram illustrating another possible application scenario of the information recognition method proposed in this application;

[0056] Figure 7 A flowchart illustrating yet another optional example of the information identification method proposed in this application;

[0057] Figure 8 A flowchart illustrating yet another optional example of the information identification method proposed in this application;

[0058] Figure 9 A schematic diagram of an optional example of the information identification device proposed in this application;

[0059] Figure 10 This is a schematic diagram of another alternative example of the information identification device proposed in this application. Detailed Implementation

[0060] In order to improve the accuracy and reliability of information recognition, this application proposes that, during the object recognition process of the current frame image of the acquired text to be recognized, the positional relationship between the recognition regions of the indicator object (such as a user's finger, a reading pen, etc.) and the object to be recognized (such as a single character / word in the text to be recognized) in the current frame image is detected to determine whether it is necessary to retrieve at least one adjacent historical frame image of the current frame image as the image to be recognized for recognizing the object to be recognized in the current frame image.

[0061] Because the video stream captured by the camera is continuous during text recognition, and the movement of the indicated object between adjacent objects to be recognized is continuous, the image to be recognized contains the current object to be recognized, and the area of ​​the object to be recognized occluded by the indicated object in the image to be recognized is small or not occluded. Thus, by performing recognition processing on the image to be recognized, the required recognition processing result can be accurately obtained, such as playing the recognized object to be recognized by voice, or translating and broadcasting the object to be recognized, which can be determined according to the application scenario of the text to be recognized.

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] Reference Figure 1 This is a schematic diagram of the hardware structure of an optional example of a computer device applicable to the information recognition method proposed in this application. The computer device can be a terminal or a server. The terminal may include a finger-search / translation device, a smartphone, a computer, a wearable device, an augmented reality (AR) device, a virtual reality (VR) device, etc. The server may include a physical server or a cloud server, communicating with the terminal via a wired or wireless network to meet the needs of text recognition applications. The product type of the computer device performing the information recognition method in this application is not limited and can be determined according to actual application requirements.

[0064] like Figure 1 As shown, the computer device proposed in this application embodiment may include: a communication module 11, a memory 12, and a processor 13, wherein:

[0065] The number of each of the communication module 11, memory 12 and processor 13 can be at least one, and the communication module 11, memory 12 and processor 13 can all be connected to a communication bus to communicate through the communication bus. This application does not limit the communication connection method between the various components of the computer device.

[0066] The communication module 11 may include a wireless communication module and / or a wired communication module, such as a WIFI module, a 5G / 6G (fifth-generation mobile communication network / sixth-generation mobile communication network) module, a GPRS module, a GSM module, or one or more combined communication modules. Depending on communication requirements, it may also include wired interfaces such as USB interfaces, serial / parallel ports, and multimedia communication interfaces to enable wired connections within the computer device and / or with local devices, meeting corresponding communication needs. This application does not provide detailed examples of the composition and working principle of the communication module 11; adjustments can be made to adapt it to different information identification application requirements.

[0067] The memory 12 can store a program that implements the information identification method proposed in the embodiments of this application. The processor 13 can be used to load and execute the program stored in the memory 12 to implement the various steps of the information identification method on the corresponding side of the computer device proposed in this application. The implementation process can be referred to the description of the corresponding method embodiments below, which will not be described in detail here.

[0068] The aforementioned memory 12 can also be used to cache several historical frame images adjacent to the current frame image, so as to retrieve the required historical frame images. It can also avoid storing all historical frame images, which would waste resources and reduce the performance of computer equipment. It can also store at least some recognition processing results as needed for subsequent queries. This application does not restrict the storage method of historical information such as historical frame images and historical recognition processing results.

[0069] In this embodiment, memory 12 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device. Processor 13 may be a central processing unit (CPU), application-specific integrated circuit (ASIC), digital signal processor (DSP), application-specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA), or other programmable logic devices. This application does not limit the device types of each memory 12 and processor 13 included in the computer device; they can be determined according to the corresponding functional requirements of the computer device.

[0070] In text object recognition applications, such as applications that perform finger-pointing / finding translation of the text to be recognized, such as... Figure 2As shown, the computer device may also include an image acquisition unit 14 (such as a camera) to acquire images of the text to be recognized, that is, to track and acquire images of the pointer object pointing to the text to be recognized, so as to obtain an image frame containing the pointer object and the object to be recognized it points to. Optionally, the computer device may also include at least one output component 15, such as a speaker, a display, etc., which can be determined according to functional requirements.

[0071] Taking computer devices such as finger-pointing / translation devices and point-and-read devices as examples, users can place the text to be recognized under the camera, point their finger at any object in the text, and capture the current frame image containing that content. Then, according to the information recognition method proposed in this application, the object to be recognized that the user is pointing to is accurately identified, and output directly or after translation via voice. The application process for other types of computer devices is similar, and this application will not provide examples of each. As can be seen, Figure 2 As shown, computer equipment

[0072] In some other embodiments, the image acquisition device that performs the above-described image acquisition can also be a standalone device, such as... Figure 3 In the application scenario shown, the image acquisition device sends the current frame image of the text to be recognized to a computer device, which then executes the information recognition method proposed in this application and outputs the recognition processing result. If the computer device is a terminal with an output component, the terminal can directly output the recognition processing result; if the computer device is a server, the server can send the obtained recognition processing result to a preset terminal for output. The preset terminal can be a terminal located at the site of the text to be recognized, so that the on-site user can know the recognition processing result; or it can be a remote terminal, so that a remote user can know the content of the local text to be recognized. The system architecture for the above application scenario can be determined according to the application requirements. This application does not limit the system architecture for the above application scenario.

[0073] It should be understood that, Figure 1 and Figure 2 The structure of the computer device shown does not constitute a limitation on the computer device in the embodiments of this application. In practical applications, the computer device may include more or fewer components than those shown in the above figures, or combine certain components, such as a sensor module consisting of various sensors, a power module, other input components such as microphones, etc., which will not be listed here.

[0074] Reference Figure 4 This is a flowchart illustrating an optional example of the information identification method proposed in this application. This method can be executed by the aforementioned computer device, such as... Figure 4 As shown, the method may include:

[0075] Step S41: Obtain the current frame image of the text to be recognized;

[0076] Regarding the acquisition process of the current frame image of the text to be recognized, please refer to the description of the corresponding part of the above embodiment. Therefore, the current frame image can be obtained by the image acquisition device of the computer device itself to acquire the image of the text to be recognized, or it can be acquired by other devices with image acquisition function and sent to the computer device, etc. This application does not limit the implementation method of step S41.

[0077] The text to be identified can be text content in a file, text content output by an electronic device, etc. This application does not restrict the source or form of the text to be identified, and can be determined as appropriate.

[0078] Step S42: Perform object recognition on the current frame image to determine the first recognition region of the indicated object and the second recognition region of the object to be recognized indicated by the indicated object in the text to be recognized;

[0079] In this embodiment of the application, since the image acquisition process of the text to be identified is usually continuous, the computer device may obtain video stream information. Therefore, in the continuous frame images constituting the video stream information obtained by the computer device, some frame images may only contain the text to be identified and not the indicator object, or the indicator object is in the process of moving and does not indicate any object to be identified. For such frame images, this application may temporarily not perform identification processing, but instead perform subsequent processing on the current frame image that contains the text to be identified and the indicator object, and the indicator object currently indicates any object to be identified.

[0080] In the process of image recognition, this application can use artificial intelligence image recognition algorithms (such as object detection algorithms, machine learning algorithms, etc.) to accurately identify the first recognition area of ​​the indicator object contained in the current frame image, and the second recognition area of ​​the object to be recognized indicated by the indicator object in the text to be recognized. The implementation process is not described in detail in the embodiments of this application.

[0081] It should be noted that this second recognition area contains the complete object to be recognized, that is, such as Figure 5 The image recognition result diagram shows that when the indicated object partially occludes the object to be recognized, the second recognition area of ​​the object to be recognized can be determined by combining the recognition areas of adjacent objects. This ensures that the second recognition area includes not only the unoccluded area of ​​the object to be recognized (denoted as the display area) but also the occluded area occluded by the indicated object. This ensures that the positional relationship between these two recognition areas can be used to determine whether the object to be recognized is occluded by the indicated object, but it is not limited to this recognition method.

[0082] In the process of detecting the first recognition area of ​​the indicated object, as shown in 5, the recognition area of ​​a preset size or proportion can be determined as the first recognition area based on the vertex position (such as the fingertip position) of the recognized indicated object in the direction of indication and the left and right edge positions of the object to be recognized. That is to say, the first recognition area does not need to include the entire indicated object, but only needs to include the relevant information of the indication purpose of the indicated object. This application does not restrict the method of obtaining the first recognition area.

[0083] Step S43: If the second recognition region is detected to be occluded by the first recognition region and the preset conditions are met, the historical frame image of at least one adjacent frame of the current frame image is obtained as the image to be recognized.

[0084] In practical applications, it is determined that there is an overlapping area between the first recognition area and the second recognition area according to the above method. That is, at least part of the second recognition area is occluded by the first recognition area, which may affect the recognition accuracy of the object to be recognized in the second recognition area. In this case, this application can further determine whether the preset conditions are met to determine whether to retrieve the historical frame image of at least one adjacent frame of the current frame image to realize the recognition processing of the object to be recognized.

[0085] The preset conditions may include, but are not limited to, setting conditions for parameters related to the overlapping area between the first and second recognition areas (i.e., the occluded area where the second recognition area is occluded by the first recognition area), or setting conditions for parameters related to the display area where the second recognition area is not occluded by the first recognition area. The preset conditions may include one or more conditions. This application does not limit the contents of the preset conditions.

[0086] Based on the above analysis, if the preset conditions are met, it can be considered that the occlusion of the object to be identified in the current frame image may prevent the object to be identified from being accurately identified in the current frame image. In order to ensure the reliability and accuracy of the identification processing results, this application can retrieve historical frame images as images to be identified in the above manner, instead of using the current frame image as the image to be identified to perform subsequent identification processing steps.

[0087] It should be understood that, during the process of the pointer continuously pointing to different objects on the text to be recognized, the positional relationship between the pointer and the currently pointed object is not limited to, for example, Figure 5 In some embodiments, the indicated object may not obscure the object to be identified, such as when the indicator vertex is located below the second identification region and the first and second identification regions do not overlap. In this case, the object to be identified indicated by the indicator object in the current frame image is clear, and the acquired current frame image can be directly determined as the image to be identified for subsequent identification processing.

[0088] Step S44: Perform recognition processing on the objects to be recognized contained in the image to be recognized, and output the recognition processing result.

[0089] Following the analysis above, this application can utilize artificial intelligence technology that matches the current application requirements to analyze the image to be identified, which is composed of at least one historical frame image. This analysis determines the object to be identified indicated by the pointer in the current frame image, i.e., the object that the user currently needs to identify. For example, based on information such as the movement trajectory of the pointer, the object to be identified previously indicated by the pointer, and the positional relationships between objects contained in the text to be identified, the recognition area containing the complete content of the object to be identified in the current frame image can be located. In this way, the object to be identified contained in the recognition area can be recognized and processed to obtain the corresponding recognition result.

[0090] Optionally, this application can perform semantic recognition on the aforementioned object to be recognized (i.e., the target object indicated by the indicator object in the current frame image) contained in the image to be recognized. Based on the recognition business requirements, the application can obtain target speech that is consistent with the content of the object to be recognized and belongs to a specified language category, and output the target speech. Based on this, this application can realize the point-reading function and finger translation function of the text to be recognized, meeting the corresponding application requirements.

[0091] In some embodiments, during the process of obtaining target speech that is consistent with the content of the object to be identified and belongs to a specified language category, the user can personalize the timbre, volume, and other sound features of the target speech. For example, before continuing information recognition, the user can select at least one sound feature of the output speech so that the vocoder can combine the sound feature to process the semantic recognition result of the object to be identified and obtain target speech that matches the user's personal preferences, such as using the timbre of a famous person to broadcast the content of the object to be identified or its translation. Of course, this application can also directly use the default timbre to obtain the target speech. The method of obtaining the target speech in this application is not limited and can be determined as appropriate.

[0092] It should be noted that the object to be identified above can be a text / character that the pointer vertex of the object points to or is located on, such as... Figure 5 The word "and" shown can also be a word identified by combining semantic recognition of the context, such as... Figure 5 The text to be identified shown includes words such as "research" and "artificial intelligence". The figure does not show a schematic diagram of the text area where the words are located. In this case, the second identification area mentioned above can be the text area where the words are located, especially in the context of translation business applications. The implementation process will not be described in detail in this application.

[0093] In summary, combining Figure 6 The schematic diagram of the information recognition method illustrates that when performing object recognition on the current frame image of the acquired text to be recognized and determining the second recognition region of the object to be recognized indicated by the indicator object, this application also obtains the first recognition region of the indicator object. If the second recognition region is detected to be occluded by the first recognition region and a preset condition is met, this application will obtain at least one adjacent historical frame image of the current frame image as the image to be recognized. By performing recognition processing on the aforementioned object to be recognized contained in the image to be recognized, the content of the object to be recognized can be determined with high accuracy and reliability, ensuring the accuracy of the recognition processing result obtained thereby. Compared with the method of still performing object recognition processing on the current frame image, this application improves the reliability and accuracy of the information recognition result.

[0094] Reference Figure 7 This is a flowchart illustrating another optional example of the information identification method proposed in this application. This method can still be executed by the aforementioned computer equipment, such as... Figure 7 As shown, the method may include:

[0095] Step S71: Obtain the current frame image of the text to be recognized;

[0096] Step S72: Perform object recognition on the current frame image to determine the first recognition region of the indicated object and the second recognition region of the object to be recognized indicated by the indicated object in the text to be recognized;

[0097] The implementation process of steps S71 and S72 can be referred to the description of the corresponding parts of the above embodiments, and will not be repeated in this embodiment.

[0098] Step S73: Obtain the overlapping area between the first recognition area and the second recognition area, and determine the length of the overlapping area in the specified direction;

[0099] Step S74: Detect whether the length is less than the first length threshold. If not, proceed to step S7; if yes, proceed to step S75.

[0100] Step S75: Detect whether the length is greater than the second length threshold. If not, proceed to step S77; if yes, proceed to step S76.

[0101] Step S76: Obtain at least one adjacent historical frame image of the current frame image as the image to be identified;

[0102] Based on the description of the preset conditions in the above embodiments, in order to detect whether the occlusion of the object to be identified by the indicator object in the current frame image will affect the accuracy of the recognition result, this application proposes to analyze and determine the overlapping area between the detected first recognition area and the second recognition area. As described above, this can be achieved by detecting whether the length of the text to be identified in the overlapping area in the object reading direction or in the vertical direction of the object reading direction (such as the height or width of the overlapping area) reaches the preset corresponding length threshold.

[0103] The aforementioned length threshold can be determined based on the font length (based on a preset font size) and / or width of the text to be recognized. The specific value is not limited. It can be understood that the values ​​of the first length threshold and the second length threshold are different for different specified directions, and can be determined based on configuration parameters such as the font type and size of the text to be recognized.

[0104] Since the less the object to be identified is obscured by the indicated object, the more reliable the identification result of the object to be identified in the current frame image is. Generally, the less the object to be identified is obscured, the smaller the length of the detected overlapping area in the specified direction; conversely, the more the object to be identified is obscured, the larger the length of the detected overlapping area in the specified direction. Therefore, if the length of the detected overlapping area in the specified direction is less than the first length threshold, it can be considered that the object to be identified is not completely obscured and some content is displayed, which may be correctly identified. However, there is still a probability of being misidentified. Therefore, in order to ensure the accuracy of identification, step S76 can be executed directly. Of course, step S74 can also be executed for further detection to reduce unnecessary calls to historical frame images.

[0105] If the length of the overlapping region in a specified direction is less than the first length threshold and greater than the second length threshold, it can be considered that the occluded area of ​​the object to be identified in the current frame image will likely affect the accuracy of the content recognition result of the object to be identified, or even make it impossible to identify the content of the object to be identified from the current frame image. In this case, step S76 can be executed to accurately identify the content of the object to be identified by taking advantage of the characteristic that the object to be identified is not occluded or the occluded content is very small in the historical frame image and does not affect the object content recognition.

[0106] Therefore, in the above embodiments, detecting that the second identification area is occluded by the first identification area and meets the preset conditions may include, but are not limited to: obtaining the overlapping area between the first identification area and the second identification area; detecting that the area of ​​the overlapping area is less than the first area threshold; or detecting that the area of ​​the overlapping area is less than the first area threshold and greater than the second area threshold.

[0107] In some other embodiments of this application, for the aforementioned overlapping regions, this application can also determine whether the occlusion of the object to be identified in the current frame image will affect the accuracy of the content recognition result and whether it is necessary to retrieve historical frame images by detecting whether the area of ​​the overlapping region reaches the corresponding area threshold. Based on this, the above step of detecting that the second recognition region is occluded by the first recognition region and meets the preset conditions may also include detecting that the area of ​​the overlapping region is less than the first area threshold; or, detecting that the area of ​​the overlapping region is less than the first area threshold and greater than the second area threshold, and executing step S76. The implementation process is not detailed in this application. Among them, the above-mentioned area thresholds can be determined based on the area of ​​the complete text region of the object to be identified, and can represent the critical value of whether the content of the object to be identified can be accurately detected. The specific values ​​are not limited.

[0108] Step S77: Determine the current frame image as the image to be recognized;

[0109] Following the above analysis, if the length of the overlapping region is less than the second length threshold and / or the area of ​​the overlapping region is less than the second area threshold, it can be considered that the content of the object to be identified is very small and does not affect the content recognition of the object to be identified. Therefore, in this embodiment, the current frame image can be directly used as the image to be identified for subsequent recognition processing.

[0110] Step S78: Perform recognition processing on the objects to be recognized contained in the image to be recognized, and obtain the recognition processing result;

[0111] The identification process in step S78 can be referred to the description of the corresponding part of the context embodiment, and will not be described in detail in this embodiment.

[0112] Step S79: Determine the third recognition region of the first adjacent object in the indication direction of the object to be recognized in the current frame image, as indicated by the indicating object;

[0113] Step S710: Perform recognition processing on the first adjacent object in the third recognition area to obtain the recognition processing result;

[0114] Based on the above description, if the computer device detects that the second recognition area is occluded by the first recognition area and does not meet the preset conditions, combined with the above description of the preset conditions, namely, the area of ​​the overlapping area is greater than the first area threshold, or the length of the overlapping area in the specified direction is greater than the first length threshold, it can be considered that the object to be recognized obtained from the current frame image is not the target object that the user currently wants to recognize. In other words, the object in the text to be recognized that the indicated object is should be the adjacent object of the object to be recognized in the indicated direction. It is necessary to redetermine the recognition area (i.e., the third recognition area) of the actual object to be recognized in the current frame image. The implementation process includes, but is not limited to, the method described above.

[0115] Step S711: Output the obtained recognition processing result.

[0116] It is understood that the implementation method of step S711 often differs for different types of computer devices. If the computer device is a server, the server can send the obtained recognition and processing results to a preset terminal for broadcasting; if the computer device is a local terminal, the recognition and processing results can be broadcast through a speaker, and / or the translated content of the recognition and processing results can be displayed on a monitor, etc. This application does not limit the implementation method of step S711 and can be determined according to the situation.

[0117] In some other embodiments of this application, after obtaining the first identification area and the second identification area according to the method described above, this application can also obtain the display area in the second identification area that is not obscured by the first identification area, and determine whether it is necessary to retrieve historical frame images by comparing the area of ​​the display area or the length in a specified direction with the corresponding threshold.

[0118] In practical applications, if the area of ​​the above-mentioned display area is detected to be greater than the third area threshold and less than the global area (i.e., the area occupied by the complete content of the object to be identified); or if the length of the above-mentioned display area in the specified direction is detected to be greater than the third length threshold and less than the global length (i.e., the length of the area in the specified direction where the complete content of the object to be identified is located), then based on the detection result, it can be considered that the content of the object to be identified is occluded by the indicated object. There is a probability of false detection when performing content detection on the object to be identified in the current frame image. Therefore, the content of step S76 above can be directly executed to ensure the high reliability and high accuracy of the obtained recognition result.

[0119] Optionally, to reduce the retrieval of historical frame images, this application can further refine the detection results above. For example, if the area of ​​the aforementioned display region is detected to be greater than the third area threshold and less than the fourth area threshold; or if the length of the aforementioned display region in a specified direction is detected to be greater than the third length threshold and less than the fourth length threshold, it can more accurately determine that the object to be identified in the current frame image is the object currently to be identified, and that the degree of occlusion by the indicated object will lead to inaccurate identification results for the object to be identified in the current frame image. Therefore, when obtaining this detection result, step S76 can be directly executed, and the implementation process will not be detailed in this application.

[0120] Based on the above analysis, if the area of ​​the display area is greater than the fourth area threshold or the length of the display area is greater than the fourth length threshold, it can be considered that the degree of occlusion of the object to be identified by the indicator object in the current frame image does not affect the content detection result of the object to be identified. The current frame image can be determined as the image to be identified, and the identification processing of the object to be identified can be realized.

[0121] Similarly, if the area of ​​the display area is detected to be less than the third area threshold, or the length of the display area in the specified direction is less than the third length threshold, it can be considered that the object to be identified in the current frame image obtained previously is not the actual object to be identified. Step S78 can be executed to determine the third identification area in the current frame image that is actually the object to be identified, so as to ensure the reliability and accuracy of the obtained identification processing result. The implementation process can be referred to the description of the corresponding part of the above embodiment, and will not be described in detail in this embodiment.

[0122] The thresholds for the area of ​​the display area or the length in the specified direction can be determined based on the area of ​​the entire content of the object to be identified or the length in the specified direction, and the specific values ​​are not limited.

[0123] In summary, in the above embodiments of this application, when the object to be identified indicated by the indicator object in the current frame image is occluded by the indicator object, the parameters (such as area or length in a specified direction) of the occluded area (i.e., the overlapping area) or the unoccluded area (i.e., the display area) of the object to be identified can be detected according to the method described above. The comparison result with at least one preset corresponding threshold can be used to determine whether the object to be identified is the actual object to be identified. If not, the object with complete content displayed above the indicator object in the current frame image can be considered as the object to be identified. If so, it can be determined whether the degree of occlusion of the object to be identified by the indicator object in the current frame image will lead to false detection. If so, this application can locate the text area of ​​the object to be identified in the historical frame image, accurately detect the content of the object to be identified, and obtain the required recognition processing result. This solves the technical problem that the recognition processing result is incorrect when the object to be identified is still processed in the current frame image under severe occlusion, thus improving the accuracy and reliability of information recognition.

[0124] Reference Figure 8 This is a flowchart illustrating another optional example of the information identification method proposed in this application. This method can still be executed by the aforementioned computer equipment, such as... Figure 8 As shown, the method may include:

[0125] Step S81: Obtain the current frame image of the text to be recognized;

[0126] Step S82: Perform object recognition on the current frame image to determine the first recognition region indicating the object, the object to be recognized pointed by the indicating vertex of the object, and the second adjacent object of the object to be recognized in the reading direction of the text to be recognized.

[0127] Combined with the description of the corresponding part of the above embodiments, algorithms such as image recognition, semantic analysis, and target detection of artificial intelligence can be run to achieve object recognition of the current frame image. In order to determine the recognition region where the complete content of the object to be recognized is located when the object to be recognized is blocked by the indicating object, this application can utilize the characteristic that the upper and lower boundaries of the recognition regions of adjacent objects in the text to be recognized are the same to obtain the second adjacent object of the object to be recognized in the reading direction of the text to be recognized. As Figure 5 shown, when determining the boundary of the occluded part of the recognition region of the object to be recognized, such as the character "究" blocked by a finger, the second adjacent object such as "包" or "研" adjacent to "究" and its fourth recognition region can be determined. The object recognition process of this application will not be elaborated in detail.

[0128] Step S83: Detect that the second adjacent object in the current frame image is not blocked by the indicating object, and determine the second recognition region of the object to be recognized based on the fourth recognition region of the second adjacent object.

[0129] To ensure the integrity and accuracy of the second recognition region of the obtained object to be recognized, it can be detected whether the second adjacent object is blocked by the indicating object, such as detecting whether there is an overlapping region between the fourth recognition region of the second adjacent object and the first recognition region of the indicating object. If not, the boundary of the fourth recognition region can be used to determine the second recognition region of the object to be recognized. At this time, the upper and lower edge positions of the second recognition region are the same as those of the fourth recognition region, that is, the heights of the second recognition region and the fourth recognition region are the same.

[0130] It should be noted that the method for obtaining the second recognition region of the object to be recognized includes but is not limited to the implementation methods described above. In some other embodiments, this application can also estimate the second recognition region of the object to be recognized based on the object configuration parameters of the text to be recognized, such as font size, type, etc. This application will not be elaborated one by one.

[0131] Step S84: Detect that the second recognition region is blocked by the first recognition region and meets the preset conditions, and obtain at least one adjacent historical frame image of the current frame image as the image to be recognized.

[0132] Regarding the implementation process of step S84 and the execution steps when the preset conditions are not met, reference can be made to the description of the corresponding part of the above embodiments, and this embodiment will not be elaborated.

[0133] Step S85: Track and detect the pointer object in the image to be recognized, determine the object to be recognized that the pointer object points to in the current frame image, and the fifth recognition area of ​​the object to be recognized in the image to be recognized;

[0134] As described in the corresponding section of the above embodiment, according to the above detection method, the degree of occlusion of the object to be identified by the indicator object in the current frame image is determined. To improve the accuracy of the identification processing of the object to be identified in the current frame image, it is proposed to retrieve at least one adjacent historical frame image of the current frame image as the image to be identified. The implementation process will not be described in detail.

[0135] Subsequently, since the content of the indicator objects and text objects contained in adjacent frames of images acquired by the image acquisition device is basically the same, the content of the object to be identified indicated by the indicator object in the current frame image is determined from the historical frame images by tracking and detecting the indicator objects. The implementation process is not described in detail.

[0136] Step S86: Perform semantic recognition on the object to be recognized contained in the fifth recognition region to obtain the semantic recognition result;

[0137] Step S87: Using the semantic recognition result, obtain the target speech that meets the recognition processing requirements and is semantically consistent with the object to be recognized;

[0138] The implementation process of steps S86 and S87 can be referred to, but is not limited to, the relevant description above regarding the implementation process of recognizing the object contained in the image to be recognized. The recognition processing requirements may include requirements for the language category of the output speech, such as using a default language category to broadcast the content of the object to be recognized, or using a user-specified speech category or even a specified timbre to broadcast the content of the object to be recognized, thus obtaining rich and personalized target speech. The implementation process will not be detailed in this application.

[0139] Step S88: Output the target speech.

[0140] As analyzed above, the server can send the obtained target speech to a preset terminal, and the preset terminal can output the target speech through its speaker; or, if the query / translation terminal obtains the target speech according to the above information recognition method, it can directly broadcast it through its own speaker to meet the needs of business applications. This application does not restrict the output method of the target speech.

[0141] In summary, when the degree of occlusion of the target object in the current frame image interferes with the accuracy of the target object recognition in the current frame image, this application proposes to utilize the content association characteristics between consecutive frames in a video stream to determine the recognition region of the target object from historical frames. Based on the semantic recognition results of its content, target speech that is semantically consistent with the target object is obtained, and the target speech is broadcast according to a preset language category, which greatly improves the accuracy and reliability of information recognition.

[0142] Reference Figure 9 The diagram below shows an optional example of the information identification device proposed in this application. The device may include:

[0143] Image acquisition module 91 is used to acquire the current frame image of the text to be recognized;

[0144] Object recognition module 92 is used to perform object recognition on the current frame image, determine a first recognition region of the indicated object and a second recognition region of the object to be recognized indicated by the indicated object in the text to be recognized;

[0145] The image acquisition module 93 is used to detect that the second recognition region is occluded by the first recognition region and meets the preset conditions, and to acquire at least one adjacent historical frame image of the current frame image as the image to be recognized.

[0146] The recognition processing module 94 is used to perform recognition processing on the object to be recognized contained in the image to be recognized, and output the obtained recognition processing result.

[0147] Optionally, the above-mentioned image acquisition module 93 may include the following first acquisition unit and at least one detection unit:

[0148] The first obtaining unit is used to obtain the overlapping area between the first identification area and the second identification area;

[0149] The first detection unit is used to detect that the area of ​​the overlapping region is less than a first area threshold.

[0150] The second detection unit is used to detect that the area of ​​the overlapping region is less than a first area threshold and greater than a second area threshold.

[0151] The third detection unit is used to detect that the length of the overlapping region in a specified direction is less than a first length threshold; the specified direction refers to the object reading direction of the text to be identified or the direction perpendicular to the object reading direction;

[0152] The fourth detection unit is used to detect that the length of the overlapping region in a specified direction is less than a first length threshold and greater than a second length threshold.

[0153] In some other embodiments, the image acquisition module 93 described above may include the following second acquisition unit and at least one detection unit:

[0154] The second obtaining unit is used to obtain the display area in the second identification area that is not obscured by the first identification area;

[0155] The fifth detection unit is used to detect that the area of ​​the display area is greater than the third area threshold and less than the fourth area threshold;

[0156] The sixth detection unit is used to detect that the area of ​​the display area is greater than the third area threshold.

[0157] The seventh detection unit is used to detect that the length of the display area in the specified direction is greater than a third length threshold and less than a fourth length threshold;

[0158] The eighth detection unit is used to detect that the length of the display area in the specified direction is greater than a third length threshold.

[0159] Based on the above analysis, in some other embodiments, the image acquisition module 93 of the above-mentioned device may further include:

[0160] The first determining unit is configured to determine the current frame image as an image to be identified if the overlapping region is obtained and the area of ​​the overlapping region is detected to be less than the second area threshold, or the length of the overlapping region is less than the second length threshold.

[0161] The second determining unit is configured to determine the current frame image as an image to be identified if the display area is obtained and the area of ​​the display area is detected to be greater than the fourth area threshold, or the length of the display area is greater than the fourth length threshold.

[0162] In some other embodiments, such as Figure 10 As shown, the above-mentioned device may further include:

[0163] The third identification region determination module 95 is used to detect that the second identification region is occluded by the first identification region and does not meet the preset condition, and determine the third identification region of the first adjacent object located in the indication direction of the object to be identified in the current frame image indicated by the indicator object;

[0164] The first adjacent object recognition processing module 96 is used to recognize the first adjacent object in the third recognition area and execute the recognition processing result obtained by the step described above.

[0165] Optionally, the third identification region determination module 95 mentioned above may include:

[0166] The ninth detection unit is configured to detect, when the overlapping region is obtained, that the area of ​​the overlapping region is greater than the first area threshold, or that the length of the overlapping region in the specified direction is greater than the first length threshold.

[0167] The ninth detection unit is configured to, when the display area is obtained, detect that the area of ​​the display area is less than the third area threshold, or the length of the display area in the specified direction is less than the third length threshold.

[0168] Based on the description of the above embodiments, the object recognition module 92 may include:

[0169] The first object recognition unit is used to perform object recognition on the current frame image, determine the first recognition area of ​​the indicated object, the object to be recognized pointed to by the indicator vertex of the indicated object, and the second adjacent object of the object to be recognized in the reading direction of the text to be recognized.

[0170] The second identification region determination unit is used to detect that the second adjacent object in the current frame image is not occluded by the indicated object, and to determine the second identification region of the object to be identified based on the fourth identification region of the second adjacent object; the upper and lower edges of the second identification region and the fourth identification region are consistent.

[0171] Optionally, the aforementioned identification processing module 94 may include:

[0172] The tracking and detection unit is used to track and detect the indicator object in the image to be identified, and determine the object to be identified indicated by the indicator object in the current frame image, in the fifth recognition region of the image to be identified;

[0173] A semantic recognition unit is used to perform semantic recognition on the object to be recognized contained in the fifth recognition region;

[0174] The target speech acquisition unit is used to acquire target speech that meets the recognition processing requirements and is semantically consistent with the object to be recognized; the recognition processing requirements include requirements for the language category of the output speech.

[0175] A target speech output unit is used to output the target speech.

[0176] It should be noted that the various modules and units in the above-mentioned device embodiments can all be stored in the memory as program modules. The processor executes the above-mentioned program modules stored in the memory to realize the corresponding functions. The functions realized by each program module and its combination, as well as the technical effects achieved, can be referred to the description of the corresponding part of the above-mentioned method embodiments. This embodiment will not repeat them here.

[0177] This application also provides a computer-readable storage medium on which a computer program can be stored, which can be invoked and loaded by a processor to implement the various steps of the information reporting method described in the above embodiments.

[0178] Finally, it should be noted that, regarding the above embodiments, unless the context explicitly indicates an exception, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, and these steps and elements do not constitute an exclusive list; the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0179] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more.

[0180] The terms used in this application, such as "first" and "second," are for descriptive purposes only, used to distinguish one operation, unit, or module from another, and do not necessarily require or imply any such actual relationship or order between these units, operations, or modules. Furthermore, they should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0181] Furthermore, the various embodiments in this specification are described in a progressive or parallel manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus and computer devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0182] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An information identification method, the method comprising: Obtain the current frame image of the text to be recognized; Perform object recognition on the current frame image to determine a first recognition region of the indicated object and a second recognition region of the object to be recognized indicated by the indicated object in the text to be recognized; If the second recognition region is detected to be occluded by the first recognition region and a preset condition is met, the historical frame image of at least one adjacent frame of the current frame image is obtained as the image to be recognized. The object to be identified contained in the image to be identified is identified, and the identification processing result is output; wherein, the identification region containing the complete content of the object to be identified in the current frame image contained in the image to be identified is located, the object to be identified contained in the identification region is identified, and the identification processing result is obtained.

2. The method according to claim 1, wherein detecting that the second identification area is occluded by the first identification area and meets a preset condition includes: Obtain the overlapping area between the first identification area and the second identification area; The area of ​​the overlapping region is detected to be less than a first area threshold. or, The area of ​​the overlapping region is detected to be less than a first area threshold and greater than a second area threshold; or, The overlapping region is detected to have a length less than a first length threshold in a specified direction; the specified direction refers to the object reading direction of the text to be identified or the direction perpendicular to the object reading direction; or, The length of the overlapping region in a specified direction is detected to be less than a first length threshold and greater than a second length threshold.

3. The method according to claim 2, wherein detecting that the second identification area is occluded by the first identification area and meets a preset condition further includes: Obtain the display area in the second identification region that is not obscured by the first identification region; The area of ​​the display area is detected to be greater than a third area threshold and less than a fourth area threshold. or, The area of ​​the display area is detected to be greater than a third area threshold; or, It was detected that the length of the display area in the specified direction is greater than a third length threshold and less than a fourth length threshold; or, It was detected that the length of the display area in the specified direction is greater than a third length threshold.

4. The method according to claim 3, further comprising: If the overlapping region is obtained, and it is detected that the area of ​​the overlapping region is less than the second area threshold, or the length of the overlapping region is less than the second length threshold, then the current frame image is determined to be an image to be identified. If the display area is obtained and it is detected that the area of ​​the display area is greater than the fourth area threshold, or the length of the display area is greater than the fourth length threshold, then the current frame image is determined to be an image to be identified.

5. The method according to claim 3, further comprising: If the second recognition region is detected to be occluded by the first recognition region and does not meet the preset condition, the third recognition region of the first adjacent object located in the indication direction of the object to be recognized, as indicated by the indication object in the current frame image, is determined; The first adjacent object in the third identification region is identified, and the identification result obtained by the output step is executed.

6. The method according to claim 5, wherein detecting that the second identification area is occluded by the first identification area and does not meet the preset condition includes: If the overlapping region is obtained, it is detected that the area of ​​the overlapping region is greater than the first area threshold, or the length of the overlapping region in the specified direction is greater than the first length threshold; If the display area is obtained, it is detected that the area of ​​the display area is less than the third area threshold, or the length of the display area in the specified direction is less than the third length threshold.

7. The method according to any one of claims 1-3, wherein performing object recognition on the current frame image to determine a first recognition region indicating an object and a second recognition region of the object to be recognized indicated by the indicating object in the text to be recognized comprises: Object recognition is performed on the current frame image to determine the first recognition region of the indicated object, the object to be recognized pointed to by the indicator vertex of the indicated object, and the second adjacent object of the object to be recognized in the reading direction of the text to be recognized; If the second neighboring object in the current frame image is not occluded by the indicated object, the second recognition region of the object to be recognized is determined based on the fourth recognition region of the second neighboring object; the upper and lower edges of the second recognition region are consistent with those of the fourth recognition region.

8. The method according to any one of claims 1-3, wherein the step of performing recognition processing on the object to be recognized contained in the image to be recognized and outputting the obtained recognition processing result includes: The indicated object in the image to be identified is tracked and detected, and the object to be identified indicated by the indicated object in the current frame image is determined to be in the fifth recognition region of the image to be identified. Semantic recognition is performed on the object to be recognized contained in the fifth recognition region to obtain target speech that meets the recognition processing requirements and is semantically consistent with the object to be recognized; the recognition processing requirements include language category requirements for the output speech; Output the target speech.

9. An information identification device, the device comprising: The image acquisition module is used to acquire the current frame image of the text to be recognized; The object recognition module is used to perform object recognition on the current frame image, determine a first recognition region of the indicated object and a second recognition region of the object to be recognized indicated by the indicated object in the text to be recognized; The image acquisition module is used to detect that the second recognition region is occluded by the first recognition region and meets a preset condition, and to acquire at least one adjacent historical frame image of the current frame image as the image to be recognized. The recognition processing module is used to perform recognition processing on the object to be recognized contained in the image to be recognized, and output the obtained recognition processing result; wherein, the recognition region containing the complete content of the object to be recognized identified in the current frame image contained in the image to be recognized is located, the object to be recognized contained in the recognition region is recognized, and the recognition processing result is obtained.

10. A computer device, the computer device comprising: The communication module is used to acquire the current frame image of the text to be recognized captured by the image acquisition device; A memory for storing a program that implements the information recognition method as described in any one of claims 1-8; A processor is configured to load and execute the program stored in the memory to implement the information recognition method as described in any one of claims 1-8.

Images