Image processing method, device, electronic device, storage medium, and computer program

Abstract

An image processing method, device, electronic device, and storage medium are disclosed, which include, when a trigger operation on a display interface is detected, activating an imaging device and capturing a processing target image including a target object based on the imaging device (S110), determining a target recognition result corresponding to the target object by analyzing and processing the processing target image (S120), and retrieving target presentation content corresponding to the target recognition result and presenting it to a target user based on the target presentation content (S130).

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application bearing application number 202210970119.6, filed with the State Intellectual Property Office of the People's Republic of China on August 12, 2022, the entire contents of which are incorporated herein by reference.

[0002] [Technical field] The present disclosure relates to the technical field of image processing, for example, to an image processing method, device, electronic device, and storage medium. [Background technology]

[0003] Image recognition technology is being increasingly widely applied in the field of image processing technology, but currently, it has the problem of low versatility due to inaccurate image recognition and the inability to provide effective presentations to users based on the recognized results. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an image processing method, device, electronic device, and storage medium for effectively recognizing collected images and providing corresponding presentation based on the recognition results. [Means for solving the problem]

[0005] An embodiment of the present disclosure includes: When a trigger operation on the display interface is detected, activating an imaging device and capturing a processing target image including a target object based on the imaging device; determining a target recognition result corresponding to the target object by analyzing and processing the target image; calling up target presentation content corresponding to the target recognition result and presenting it to the target user based on the target presentation content.

[0006] An embodiment of the present disclosure includes: an image acquisition module configured to, when detecting a trigger operation on the display interface, activate an imaging device and capture a processing target image including a target object based on the imaging device; an image recognition module configured to analyze and process the target image to determine a target recognition result corresponding to the target object; a presentation module configured to retrieve a target presentation corresponding to the target recognition result and present it to a target user based on the target presentation.

[0007] An embodiment of the present disclosure includes: one or more processors; a storage device configured to store one or more programs, The present invention further provides an electronic device that, when the one or more programs are executed by the one or more processors, causes the one or more processors to implement an image processing method according to any one of the embodiments of the present disclosure.

[0008] An embodiment of the present disclosure further provides a storage medium including computer-executable instructions that, when executed by a computer processor, perform an image processing method according to any embodiment of the present disclosure. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. [Figure 2] 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. [Figure 3] 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. [Figure 5] FIG. 1 is a block diagram illustrating the configuration of an image processing device according to an embodiment of the present disclosure. [Figure 6] 1 is a schematic configuration diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Although several examples of the present disclosure are shown in the drawings, the present disclosure can be realized in various forms and should not be construed as being limited to the examples described herein. On the contrary, these examples are provided to make the present disclosure clearer and more completely understandable. The drawings and examples of the present disclosure are merely illustrative and do not limit the scope of protection of the present disclosure.

[0011] The steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. Method embodiments may also include additional steps and / or omit the performance of steps shown. The scope of the present disclosure is not limited in this respect.

[0012] As used in this disclosure, the term "comprises" and variations thereof are open-ended inclusions, i.e., meaning "including, but not limited to." The term "based on" means "based at least in part on." The term "in one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one other embodiment," and the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms are provided in the description below.

[0013] The concepts of "first", "second", etc. referred to in this disclosure are merely intended to distinguish between devices, modules, or units, and do not limit the order or interdependence of functions performed by those devices, modules, or units.

[0014] The modifiers "a" and "a plurality" referred to in this disclosure are for purposes of illustration only and are not intended to be limiting, and should be understood as "one or more" unless the context clearly dictates otherwise.

[0015] The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only and do not limit the scope of those messages or information.

[0016] Before describing the technical means, an application scenario can be exemplarily described: For a visually impaired user, preferably a visually impaired user, who may be a blind or partially sighted user, or a user without cognitive ability, the technical means of the embodiments of the present disclosure can be used to recognize content or characters in the surrounding scene and present them to the user.

[0017] Based on the technical means of the embodiments of the present disclosure, by developing a corresponding application specifically or integrating the technical means into a mobile terminal device, it is possible to achieve the effect of providing services to users quickly and conveniently.

[0018] 1 is a flowchart of an image processing method according to an embodiment of the present disclosure, which is applied to a scene in which image content recognition is performed on collected image content and the image is presented to a user based on the recognition result. The method may be executed by an image processing device, which may be realized in software and / or hardware, and preferably, by an electronic device, which may be a mobile terminal, a personal computer (PC), a server, etc. Execution of the image processing method according to the embodiment of the present disclosure may be executed by a server, a client, or a client and server working together.

[0019] As shown in FIG. 1, the method includes the following steps:

[0020] In S110, when a trigger operation on the display interface is detected, the imaging device is started and an image to be processed including the target object is captured based on the imaging device.

[0021] The target object may be any object or scene to be recognized in a real scene, and the processing target image may be an image including the target object captured by an imaging device. The trigger operation may be triggering a control in a display interface or a region of a terminal device. The imaging device may be a camera or the like provided on the terminal device.

[0022] For example, for a smart terminal device, a trigger control can be installed on the display interface of the mobile terminal or the display interface of an application. When it is detected that a user triggers the trigger control, an imaging device built into the terminal device can be automatically called to capture an image to be processed. For a terminal device with a function key, the imaging device can be called by triggering one of the function keys on the terminal device. For example, the function key may include a power key, a dial key, or a call start key. After the imaging device is started, the target object to be recognized is captured by the imaging device, and the captured image including the target object can be used as the image to be processed.

[0023] For a mobile terminal device, in order to improve the startup efficiency of the imaging device of the mobile terminal device, a certain area of ​​the mobile terminal can be set as a hot zone, for example, the back of the mobile terminal can be set as a hot zone, and when the mobile terminal is in a sleep state, if the user taps the hot zone multiple times in succession, the imaging device of the mobile terminal can be started up and the image to be processed can be captured.

[0024] Based on the above technical means, the entire display interface corresponds to one operation control.

[0025] In practical application, to achieve the effect of quickly calling the imaging device, an operation control can be set on the entire display interface, that is, the entire display is an operation hot zone, and when a tap on the display is detected, it indicates that the imaging device needs to be triggered. A corresponding application can be developed to set the main interface of the application as an operation hot zone, and when a trigger operation on the operation hot zone is detected, it indicates that the imaging device needs to be called. In this embodiment, there are several ways to start the imaging device when a trigger operation on the display interface is detected, and this embodiment describes different ways.

[0026] The first may be to activate the imaging device when a single tap operation on the display interface is detected.

[0027] For example, when a single tap operation by the user on the display interface is detected, a launch control of the imaging device can be triggered to launch the imaging device.

[0028] The second is to start up the imaging device when a double tap operation on the display interface is detected.

[0029] To avoid the user from accidentally touching, a trigger method for double tap operation can be set, and when the user double taps the display interface, it indicates that an imaging device needs to be called to capture the image to be processed, and the imaging device can be activated to capture the image to be processed.

[0030] The third is to activate the imaging device when it is detected that the pressing time on the display interface reaches a predetermined time threshold.

[0031] The predetermined time threshold may be understood as a preset time for determining whether to activate the imaging device. When the interface detects that the pressing time exceeds the predetermined time, it indicates that the imaging device needs to be turned on. For example, if the predetermined time is 2S, when the display interface detects that the pressing time exceeds 2S, the imaging device is activated; when it is less than 2S, the imaging device is not activated.

[0032] The fourth is to activate the imaging device when it is detected that the touch trajectory on the display interface satisfies a predetermined trajectory.

[0033] The predetermined trajectory refers to the touch trajectory on the display interface. For example, if the predetermined trajectory is an "S" shape or a "Z" shape, and it is detected that the user's touch trajectory on the display interface is the same as the predetermined trajectory, it indicates that the imaging device needs to be started, and the imaging device is started.

[0034] Fifth, activating the imaging device when detecting that the hand posture relative to the display interface corresponds to at least one predetermined posture.

[0035] The predetermined posture may be understood as a preset gesture action or a motion posture of the terminal device.

[0036] For example, a gyroscope is typically provided in the terminal device, and the movement trajectory of the terminal device can be determined based on the gyroscope during the terminal device's movement. If the action corresponding to the movement trajectory matches a predetermined posture, the imaging device is activated. Based on the motion information of the terminal device, the posture of the hand holding the terminal device can be determined. For example, the hand posture may be lifting or lowering. If the hand posture corresponds to a posture with the arm raised, it indicates that the imaging device needs to be activated.

[0037] In S120, the processing target image is analyzed to determine a target recognition result corresponding to the target object.

[0038] The target recognition result is related to a target object in the processing target image, and for example, if the target object is a traffic light, the target recognition result may be the state of the traffic light. Preferably, the state of the traffic light may be green, red, or yellow.

[0039] For example, after collecting the target image, the target image may be analyzed based on a client, or the target image may be transmitted to a corresponding server, which analyzes and processes the target image to obtain a target recognition result corresponding to the target object. The analysis may be performed based on a pre-constructed neural network model, and the lighting status of a traffic signal may be effectively recognized based on the neural network model. For example, the target image may be an image including a traffic signal, and the analysis may be performed on the target image to determine whether the current traffic signal is red or green, and whether the traffic signal is open to traffic based on the recognition result.

[0040] In S130, the target presentation content corresponding to the target recognition result is retrieved and presented to the target user based on the target presentation content.

[0041] The target presentation content refers to the content that needs to be presented to the target user, and different target recognition results correspond to different presentation content, for example, when the target recognition result is A, the corresponding presentation content is content 1, and when the target recognition result is B, the corresponding presentation content is content 2. The target user may be understood as a user who is currently using the image processing device or a user who triggers the display interface.

[0042] For example, presentation contents corresponding to different recognition results may be stored in advance, and after a target recognition result is determined, a target presentation content matching the target recognition result is retrieved from memory based on the target recognition result, and corresponding presentations can be made to the user based on the target presentation content by audio broadcast or vibration.

[0043] Illustratively, the recognition result A corresponds to presentation content 1, and the recognition result B corresponds to presentation content 2. If it is determined that the target recognition result is A, presentation content 1 can be called up and the text corresponding to presentation content 1 can be broadcast by audio broadcast to be presented to the target user.

[0044] The technical means of the embodiments of the present disclosure, when detecting a trigger operation on a display interface, activates an imaging device, captures a processing target image including a target object based on the imaging device, analyzes and processes the processing target image to determine a target recognition result corresponding to the target object, and then retrieves target presentation content corresponding to the target recognition result and presents it to the target user based on the target presentation content, thereby solving the problem of inaccurate image recognition and ineffective presentation based on the recognition result, resulting in low versatility; after effectively recognizing the image content, the corresponding presentation content can be determined based on the recognition result and presented to the user based on the presentation content, thereby achieving the effect of improving the effectiveness and versatility of presentation.

[0045] 2 is a flowchart of an image processing method according to an embodiment of the present disclosure. Based on the above-described embodiment, the timing of detecting a trigger operation on a display interface is different. This embodiment can explain when a trigger operation on a display interface is detected. For the embodiment, reference can be made to the technical means of this embodiment. Technical terms that are the same as or correspond to those in the above embodiment will not be described here.

[0046] As shown in FIG. 2, the method includes the following steps:

[0047] In S210, the timing for detecting the trigger operation of the target user on the display interface is determined.

[0048] Preferably, current location information of the target user is obtained, and if the current location information corresponds to at least one predetermined location information, a trigger operation of the target user on the display interface is detected.

[0049] The current location information may be understood as the current location of the target user. Typically, the terminal device is equipped with a positioning module, and the current location information of the target user can be determined based on the positioning module. The predetermined location information may be understood as a geographical location previously determined or set by a developer, and the predetermined location information may be one or more. Preferably, the predetermined location may be longitude and latitude information corresponding to a traffic light intersection. For a visually impaired user, how to safely pass through the traffic light is important, and the current location information corresponding to the visually impaired user can be obtained. Based on the current location information and the predetermined location information, it is determined whether to detect a trigger operation on the display interface.

[0050] For example, multiple predetermined locations can be preset. For example, multiple traffic light intersection locations can be determined, and the locations of the traffic light intersections can be set as the predetermined locations. If the current location information corresponds to a location within an adjacent area range of the predetermined location, it can be determined that the target user is located at the traffic light intersection, and it can be detected whether the target user performs a corresponding trigger operation on the display interface. This setting can avoid false recognition and false trigger situations, and improve the accuracy of calling the imaging device and the effectiveness of image collection.

[0051] It may be understood that the target user's current location coordinate information can be acquired in real time to timely and accurately detect the target user's trigger operation on the display interface and capture and recognize images. Typically, the target user's current location information can be acquired based on a positioning module in the mobile terminal. The acquired current location information can be compared with at least one predetermined location information, for example, to determine whether the current location information and the one predetermined location information overlap or are close to each other. If the current location information and the one predetermined location information overlap or are close to each other, it can be determined that the current location information and the one predetermined location information correspond to each other, and in this case, the target user's trigger operation on the display interface is detected.

[0052] Based on the above technical means, if it is detected that the current position information of the target moving object corresponds to the target position information of the target user, a trigger operation of the target user on the display interface is detected.

[0053] To improve the versatility of applications in embodiments of the present disclosure, not only static objects but also dynamic moving objects can be recognized based on the application. For example, in the case of online car booking, some visually impaired users may not be able to recognize whether the current vehicle is the reserved target vehicle and may end up boarding the wrong vehicle. Therefore, the technical means according to embodiments of the present disclosure can determine whether a passing vehicle is the target vehicle, i.e., determine the target recognition result. Any entity that has a relative displacement with respect to the target user can be considered as the target moving object. Typically, the target moving object will be different in different application scenarios. For example, in the case of online car booking, the target moving object may be understood as the target vehicle currently reserved by the target user.

[0054] For example, after the target object and the target user establish a reservation relationship, the target object can transmit its location information to the terminal device used by the target user through a communication network, and the terminal device can compare the location information of the target object with the target location information of the target user in real time. If the current location information of the target object corresponds to the target location information, the distance between the current location information of the target object and the target location information may be less than 20 m. In this case, the target user can be notified by voice or vibration that the target object is close to the current location and can trigger a display interface to capture an image to be processed including the target object.

[0055] In S220, when a trigger operation on the display interface is detected, the imaging device is started and an image to be processed including the target object is captured based on the imaging device.

[0056] Based on the above technical means, before detecting the target user's trigger operation on the display interface, the method further includes: launching a target application and entering a main interface of the target application; and launching an imaging device when detecting the target user's trigger operation on the main interface.

[0057] The target application may be understood as a specially developed application implementing an embodiment of the present disclosure, and the main interface corresponds to the display interface.

[0058] For example, before detecting a trigger operation of the target user on the display interface, a target application can be launched to enter a main interface, i.e., a display interface, of the target application, and after detecting a trigger operation on the display interface, an imaging device can be launched to capture a processing target image including the target object based on the imaging device.

[0059] Based on the above technical means, after launching the target application, the method further includes: providing an application launch prompt to the target user, and performing a trigger operation on a display interface based on the application launch prompt.

[0060] The application activation prompt includes a sound prompt and / or a vibration prompt, and the content of the sound prompt includes a trigger operation content for activating the imaging device and / or a content prompt for entering a startup interface.

[0061] For example, for a visually impaired user, the user may initiate a touch operation before the application has been launched, making it impossible to trigger the display interface of the application. Therefore, after the application is launched, a voice prompt or vibration prompt can be given to the target user to effectively trigger the display interface, thereby achieving the effect of activating the imaging device. Preferably, after the application is launched, the voice prompt may be "Welcome, the XX application has been launched. Please tap the display interface to take a photo." After the application is launched, the terminal device may vibrate to notify the user that the application has been launched.

[0062] In S230, the processing target image is analyzed to determine a target recognition result corresponding to the target object.

[0063] In S240, target presentation content corresponding to the target recognition result is retrieved and presented to the target user based on the target presentation content.

[0064] The technical means of the embodiments of the present disclosure can determine the detection timing of the target user's trigger operation on the display interface according to different application scenarios, thereby effectively obtaining the target user's trigger operation on the display interface, effectively and conveniently calling the corresponding imaging device, achieving the effect of capturing an image to be processed including the target object, and improving the effectiveness of image capture.

[0065] 3 is a flowchart of an image processing method according to an embodiment of the present disclosure. Based on the above embodiment, a pre-trained image processing model can be used to recognize target objects in a processing target image, determine target recognition results corresponding to different target objects, and present the results based on the target recognition results. The embodiment can refer to the technical solutions of this embodiment. Technical terms that are the same as or correspond to those of the above embodiment will not be described here.

[0066] As shown in FIG. 3, the method includes the following steps:

[0067] In S310, when a trigger operation on the display interface is detected, the imaging device is started and an image to be processed including the target object is captured based on the imaging device.

[0068] In S320, the target object in the processing target image is recognized based on the pre-trained image processing model, and a target recognition result corresponding to the target object is obtained.

[0069] The image processing model may be pre-trained and determines target recognition results corresponding to the image to be processed.

[0070] A large number of images and corresponding recognition results can be used as a training set in advance, and a model for determining a target recognition result corresponding to a processing object can be trained as an image processing model based on the training set.

[0071] Illustratively, a target image is used as an input of an image processing model, and the image processing model can process the target image to recognize a target object in the target image, and finally, the image processing model can output a target recognition result corresponding to the target object in the target image.

[0072] Based on the above technical means, the target object is a traffic signal, and recognizing the target object in the processing target image based on the pre-trained image processing model and obtaining a target recognition result corresponding to the target object includes inputting the processing target image into the image processing model, determining a traffic state corresponding to the traffic signal based on the image processing model, and setting the traffic state as the target recognition result.

[0073] The passing state includes a passable state or an impassable state.

[0074] For example, if the target object is a traffic light, the image to be processed can be input to the image processing model, and the image processing model can recognize the traffic light in the image to be processed, i.e., recognize whether the current light is red or green, and further output a traffic status corresponding to the color of the traffic light, with the traffic status being the target recognition result. Preferably, if the traffic light in the image to be processed is red, the output result of the image processing model is a no-pass state, and conversely, if the traffic light in the image to be processed is green, the output result of the image processing model is a pass-through state.

[0075] Based on the above technical means, the target object is a target vehicle or text information, and recognizing the target object in the processing target image based on a pre-trained image processing model to obtain a target recognition result corresponding to the target object includes recognizing the vehicle information or text content of the target object in the processing target image based on the image processing model to obtain a target recognition result corresponding to the target object.

[0076] For example, if the target object is a target vehicle, vehicle information in the processing target image can be recognized based on the image processing model, for example, the vehicle model, color, license plate, etc., and the recognized vehicle model information, color information, and license plate information can be used as the target recognition result. If the target object is text information, the text information can be captured to obtain a processing target image containing the text information, and the image can be input into the image recognition model, which can recognize the text in the image and use the recognized text content as the target recognition result.

[0077] In S330, target presentation content corresponding to the target recognition result is retrieved and presented to the target user based on the target presentation content.

[0078] The technical means according to the embodiments of the present disclosure can recognize and process a target object in a processing target image based on an image processing model, determine target presentation content based on the target recognition result, and then present it to the target object based on the target presentation content, thereby not only improving the accuracy of recognition but also improving the effectiveness of presentation.

[0079] 4 is a flowchart of an image processing method according to an embodiment of the present disclosure. Based on the above embodiment, in order to improve the accuracy of the recognition result, the target recognition result can be determined by combining with the recognition result of another terminal device. The embodiment can refer to the technical means of this embodiment. The same or corresponding technical terms as those in the above embodiment will not be described here.

[0080] As shown in FIG. 4, the method includes the following steps:

[0081] In S410, when a trigger operation on the display interface is detected, the imaging device is started and an image to be processed including the target object is captured based on the imaging device.

[0082] Based on the above technical means, after starting the imaging device and entering the imaging interface, the method includes: The method further includes returning to the display interface and playing back presentation information for returning to the display interface when detecting that a trigger operation on the imaging interface satisfies a condition for returning from the imaging interface to the display interface.

[0083] For example, the imaging interface can be entered after starting the imaging device, and after the imaging is completed or when the user wants to stop the imaging, a trigger operation can be performed on the imaging interface to return from the imaging interface to the display interface. The condition for returning from the imaging interface to the display interface can be a swipe operation or a double tap operation on the imaging interface. When one or more of the above operations are detected, the imaging interface can return to the display interface, and audio can be played and presented.

[0084] In S420, obtain a use object recognition result corresponding to the target object determined by at least one terminal device.

[0085] The terminal device may be a device for recognizing or capturing an image of a target object, and in this case, the terminal device is distinguished from the terminal device used by the target user. The number of terminals may be one or more, and in order to improve the accuracy of the target recognition result, as many recognition results recognized by other terminal devices as possible may be obtained. The recognition results recognized by other terminal devices are regarded as the target recognition result.

[0086] It may be understood that in the process of determining the target recognition result of the image to be processed, the target recognition result recognized by another terminal device that matches the specified location information at the same time or within a specified interval, preferably within 300 ms, can be obtained as the target recognition result to be used.

[0087] Based on the above technical means, there are several ways to obtain the use object recognition result corresponding to the target object determined by at least one terminal device, and each way will be described below.

[0088] In order to avoid problems such as the screen contents of the image to be processed becoming unclear or the target object being unable to be effectively recognized due to differences in external light irradiation, imaging angle, etc., the target recognition result corresponding to the image to be processed can be finally determined by combining it with the recognition results of other terminal devices, thereby improving the accuracy of determining the target recognition result.

[0089] The first may be to process, based on at least one terminal device, the collected processing object image including the target object to obtain a usage object recognition result corresponding to the target object.

[0090] For example, the other user may use a corresponding terminal device to capture a processing target image including a target object, and determine a usage target recognition result corresponding to the processing target image.

[0091] The second may be determining the use object recognition result based on a use object recognition result corresponding to the target object received by at least one terminal device.

[0092] It may be understood that workers (or volunteers) can upload the lighting status of the traffic light based on corresponding application software, or obtain the lighting rules of multiple traffic lights in advance to obtain the lighting status corresponding to different times, and use the lighting status as the target recognition result.

[0093] The third may be to determine the use object recognition result based on map data information corresponding to the target user.

[0094] It may be understood that any relevant navigation software can display information such as traffic lights, and in combination with map data, can comprehensively determine the target recognition result corresponding to the target object in the current location information.

[0095] In S430, the target recognition result is updated based on the target recognition result output by the used object recognition result and the image processing model.

[0096] Illustratively, different use object recognition results and target recognition results can be assigned corresponding weight values, a new target recognition result can be determined based on the weight score of the target recognition result and the weight and score of the use object recognition result, and the previously determined target recognition result can be updated based on the new target recognition result.

[0097] In S440, the target presentation content corresponding to the target recognition result is retrieved and presented to the target user based on the target presentation content.

[0098] Based on the above technical means, the target object corresponds to a traffic signal, the target recognition result includes a passable state, and calling up the target presentation content corresponding to the target recognition result includes: determining, under a condition that the passable state is a passable state, that the target presentation content is audio information and a first vibration frequency corresponding to the passable state; and determining, under a condition that the passable state is a passable state, that the target presentation content is audio information and a second vibration frequency corresponding to the passable state.

[0099] For example, the audio information corresponding to the blocked state may be "You are currently blocked from passing, please wait a moment," and the first vibration frequency may be continuous vibration. The audio information corresponding to the passable state may be "You are currently allowed to pass, please pass as quickly as possible," and the second vibration frequency may be intermittent vibration. In this way, different target recognition results correspond to different presentation methods, and an effective presentation effect for the user can be achieved.

[0100] Based on the above technical means, presenting based on the target presentation content includes playing corresponding audio information and controlling the terminal device corresponding to the target user to vibrate at a corresponding vibration frequency.

[0101] For example, if the current road condition is impassable, the terminal device used by the user broadcasts audio information of the impassable state and vibrates continuously to warn the user not to pass through, thereby avoiding danger to the user. Correspondingly, if the current road condition is passable, the terminal device broadcasts audio to advise the user to pass through as soon as possible and vibrates at a second vibration frequency, which can be set to correspond to the pitch of most users' movements.

[0102] The technical means of the embodiments of the present disclosure can update the target recognition result by combining it with the recognition results of other terminal devices when determining the target recognition result corresponding to the target object, thereby improving the accuracy of the target recognition result.

[0103] Based on the above multiple technical means, when making presentations based on audio information, the presentations can be made using corresponding language types, such as dialects, language types corresponding to different countries, etc., to achieve the technical effect of personalized presentation.

[0104] 5 is a block diagram of an image processing device according to an embodiment of the present disclosure, which can execute an image processing method according to any embodiment of the present disclosure, and includes corresponding functional modules and effects for executing the method. As shown in FIG. 5, the device includes an image acquisition module 510, an image recognition module 520, and a presentation module 530.

[0105] the image acquisition module 510 is configured to, when detecting a trigger operation on the display interface, activate an imaging device and capture an image to be processed including a target object based on the imaging device; the image recognition module 520 is configured to analyze and process the target image to determine a target recognition result corresponding to the target object; The presentation module 530 is configured to retrieve a target presentation corresponding to the target recognition result, and present it to the target user based on the target presentation.

[0106] Based on the above device, the image processing device a current location information acquisition module configured to acquire current location information of the target user; and a current location information comparison module configured to detect a trigger operation of the target user on the display interface when the current location information corresponds to at least one predetermined location information.

[0107] Based on the above device, the image processing device The device further includes a target moving body position comparison module configured to detect a trigger operation of the target user on the display interface when it detects that the current position information of the target moving body corresponds to the target position information of the target user.

[0108] Based on the above device, the image processing device an application launching module configured to launch a target application and enter a main interface of the target application before detecting a trigger operation of the target user on the display interface, and launch the imaging device when detecting a trigger operation of the target user on the main interface; The main interface corresponds to the display interface.

[0109] Based on the above device, the image processing device an application presentation module configured to, after launching the target application, present an application launch presentation to the target user, thereby triggering a display interface based on the application launch presentation; The application startup prompt includes a sound prompt and / or a vibration prompt, and the content of the sound prompt includes a trigger operation content for starting the imaging device and / or a content prompt for entering a startup interface.

[0110] Based on the above device, the entire display interface corresponds to one operation control.

[0111] Based on the above device, the image acquisition module 510: a single tap detection module configured to wake up the imaging device when detecting a single tap operation on the display interface; a double-tap detection module configured to activate an imaging device when detecting a double-tap operation on the display interface; an imaging detection module configured to activate the imaging device when detecting that a pressing time on the display interface reaches a predetermined time threshold; a trajectory detection module configured to activate the imaging device when detecting that a touch trajectory on the display interface satisfies a predetermined trajectory; a posture detection module configured to activate the imaging device when it detects that a posture of a hand relative to the display interface corresponds to at least one predetermined posture.

[0112] Based on the above device, the image recognition module 520: and a model recognition module configured to recognize a target object in the processing target image based on a pre-trained image processing model to obtain a target recognition result corresponding to the target object.

[0113] Based on the above apparatus, the target object is a traffic light, and the model recognition module: a traffic state recognition unit configured to input the processing target image to the image processing model, determine a traffic state corresponding to the traffic signal based on the image processing model, and set the traffic state as the target recognition result; The passing state includes a passable state or an impassable state.

[0114] According to the above-mentioned device, the target object is a target vehicle or text information, and the model recognition module: The image processing system further includes a vehicle or character recognition unit configured to recognize vehicle information or character content of a target object in the processing target image based on the image processing model, and obtain a target recognition result corresponding to the target object.

[0115] Based on the above device, the image processing device a use object recognition result acquisition module configured to acquire a use object recognition result corresponding to the target object determined by at least one terminal device; The image processing system further includes a target recognition result update module configured to update the target recognition result based on the target recognition result and the target recognition result output by the image processing model.

[0116] Based on the above device, the use object recognition result acquisition module: a collection unit configured to process the collected processing object images including the target object based on at least one terminal device to obtain a usage object recognition result corresponding to the target object; a recognition unit configured to determine the target object recognition result based on a target object recognition result received by at least one terminal device, the target object recognition result being associated with the target object; and / or a determining unit configured to determine the target recognition result based on map data information corresponding to the target user.

[0117] Based on the above device, the target object corresponds to a traffic light, and the presentation module 530 a first presentation content determination unit configured to determine, under a condition that the passage state is an impassable state, that the target presentation content is audio information and a first vibration frequency corresponding to the impassable state; and a second presentation content determination unit configured to determine, under a condition that the traffic state is a passable state, that the target presentation content is audio information and a second vibration frequency corresponding to the passable state.

[0118] Based on the above device, the presentation module 530: A presentation unit configured to play corresponding audio information and control a terminal device corresponding to the target user to vibrate at a corresponding vibration frequency.

[0119] Based on the above device, the image processing device The imaging device further includes a return module configured to return to the display interface and play back presentation information that is returned to the display interface when detecting that a trigger operation in the imaging interface satisfies a condition for returning from the imaging interface to the display interface after starting the imaging device and entering the imaging interface.

[0120] The technical means of the embodiments of the present disclosure, when detecting a trigger operation on a display interface, activates an imaging device, captures a processing target image including a target object based on the imaging device, analyzes and processes the processing target image to determine a target recognition result corresponding to the target object, and then retrieves target presentation content corresponding to the target recognition result and presents it to the target user based on the target presentation content, thereby solving the problem of inaccurate image recognition and ineffective presentation based on the recognition result, resulting in low versatility; after effectively recognizing the image content, the corresponding presentation content can be determined based on the recognition result and presented to the user based on the presentation content, thereby achieving the effect of improving the effectiveness and versatility of presentation.

[0121] An image processing device according to an embodiment of the present disclosure can execute an image processing method according to any embodiment of the present disclosure, and includes corresponding functional modules and effects for executing the method.

[0122] The multiple units and modules included in the above device are merely divided according to functional logic and are not limited to the above division, as long as they can realize the corresponding functions. Furthermore, the names of the multiple functional units are merely intended to make it easier to distinguish them from one another, and do not limit the scope of protection of the embodiments of the present disclosure.

[0123] FIG. 6 is a schematic configuration diagram of an electronic device according to an embodiment of the present disclosure. Hereinafter, FIG. 6 is a schematic configuration diagram of an electronic device (e.g., a terminal device or server in FIG. 6) 600 for realizing an embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include mobile terminals such as a mobile phone, a laptop computer, a digital broadcast receiver, a personal digital assistant (PDA), a tablet computer (Portable Android Device, PAD), a portable media player (Portable Media Player, PMP), and an in-vehicle terminal (e.g., an in-vehicle navigation terminal), as well as fixed terminals such as a digital television (TV) and a desktop computer. The electronic device shown in FIG. 6 is merely an example and should not limit the functionality and scope of use of the embodiment of the present disclosure.

[0124] 6, the electronic device 600 may include a processing unit (e.g., a central processor, a graphics processor, etc.) 601, which can perform various appropriate operations and processes based on programs stored in a read-only memory (ROM) 602 or programs loaded from a storage device 608 into a random access memory (RAM) 603. The RAM 603 further stores various programs and data necessary for the operation of the electronic device 600. The processing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0125] Typically, devices connectable to the I / O interface 605 include input devices 606, including, for example, a touch panel, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607, including, for example, a liquid crystal display (LCD), speaker, oscillator, etc.; storage devices 608, including tape, hard disk, etc.; and communication devices 609. The communication devices 609 enable the electronic device 600 to communicate wirelessly or via wires with other devices to exchange data. While FIG. 6 illustrates the electronic device 600 having various devices, it should be understood that it need not embody or include all of the devices shown. More or fewer devices may alternatively be embodied or included.

[0126] According to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, embodiments of the present disclosure may include a computer program product, the computer program product being stored on a non-transitory computer-readable medium, the computer program including program code for performing the methods illustrated in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 609, installed from the storage device 608, or installed from the ROM 602. When executed by the processing device 601, the computer program performs the functions described above, which are specific to the methods of the embodiments of the present disclosure.

[0127] The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only and do not limit the scope of those messages or information.

[0128] The electronic device according to the embodiments of the present disclosure belongs to the same concept as the image processing method according to the above-mentioned embodiments, and for details of the technology not described in detail in this embodiment, the above-mentioned embodiments can be referred to, and this embodiment and the above-mentioned embodiments have the same effects.

[0129] An embodiment of the present disclosure provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the image processing method according to the embodiment is realized.

[0130] The computer-readable medium of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. The computer-readable storage medium may include an electrical connection having one or more conductors, a portable computer disk, a hard disk, RAM, ROM, an Erasable Programmable Read-Only Memory (EPROM), flash memory, optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used in or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a propagated data signal, in baseband or as part of a carrier wave, having computer-readable program code stored thereon. Such propagated data signals may take various forms, including electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which is capable of transmitting, propagating, or transmitting a program for use in or in connection with an instruction execution system, apparatus, or device. Program code contained in a computer-readable medium may be transmitted over any suitable medium, including wire, cable, radio frequency (RF), or the like, or any suitable combination of the above.

[0131] In some embodiments, the clients and servers may communicate using any network protocol now known or later developed, such as HyperText Transfer Protocol (HTTP), and may be communicatively connected (e.g., a communications network) with digital data in any form or medium. Examples of communications networks include a local area network (LAN), a wide area network (WAN), the internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), and any network now known or later developed.

[0132] The computer-readable medium may be included in the electronic device, or may exist independently without being assembled to the electronic device.

[0133] The computer-readable medium stores one or more programs, and when the one or more programs are executed by the electronic device, the electronic device When a trigger operation on the display interface is detected, the image capturing device is started and a processing target image including a target object is captured based on the image capturing device, the processing target image is analyzed and processed to determine a target recognition result corresponding to the target object, and target presentation content corresponding to the target recognition result is retrieved and presented to the target user based on the target presentation content.

[0134] Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., as well as common procedural programming languages ​​such as "C" or similar programming languages. The program code may run completely or partially on the user's computer, may run as a separate software package, may run partially on the user's computer and partially on a remote computer, or may run entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network (including a LAN or WAN) or may be connected to an external computer (e.g., over the Internet using an Internet Service Provider).

[0135] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or portion of code, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the figures. For example, two blocks shown in succession may actually be executed essentially in parallel or in the reverse order, as determined by the functionality involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or operation, or by a combination of dedicated hardware and computer instructions.

[0136] The units described in the embodiments of the present disclosure may be implemented by software or hardware. The names of the units may not necessarily be limiting, for example, the first acquiring unit may be described as "a unit for acquiring at least two Internet Protocol addresses."

[0137] The functions described herein may be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that may be used include Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Parts (ASSPs), System on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0138] In the specification of this disclosure, a machine-readable medium may be a tangible medium that contains or can store a program for use in or in connection with an instruction execution system, device, or apparatus. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. The machine-readable storage medium includes an electrical connection through one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM, or flash memory, optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination thereof. The storage medium may also be a non-transitory storage medium.

[0139] According to one or more embodiments of the present disclosure, in Example 1, When a trigger operation on the display interface is detected, activating an imaging device and capturing a processing target image including a target object based on the imaging device; determining a target recognition result corresponding to the target object by analyzing and processing the target image; calling up target presentation content corresponding to the target recognition result and presenting it to the target user based on the target presentation content.

[0140] According to one or more embodiments of the present disclosure, in Example 2, Preferably, obtaining current location information of the target user; detecting a trigger operation of the target user on the display interface when the current location information corresponds to at least one predetermined location information.

[0141] According to one or more embodiments of the present disclosure, in Example 3, Preferably, an image processing method is provided, further comprising detecting a trigger operation of the target user on the display interface when it is detected that the current position information of the target moving object corresponds to the target position information of the target user.

[0142] According to one or more embodiments of the present disclosure, in Example 4, Preferably, before detecting the trigger operation of the target user on the display interface, the method further includes: launching a target application to enter a main interface of the target application; and activating an imaging device when detecting the trigger operation of the target user on the main interface; The main interface provides an image processing method corresponding to the display interface.

[0143] According to one or more embodiments of the present disclosure, in Example 5, Preferably, after launching the target application, the method comprises: Further comprising: providing an application launch prompt to the target user, and performing a trigger operation on a display interface based on the application launch prompt; The application startup prompt includes audio prompt and / or vibration prompt, and the content of the audio prompt includes trigger operation content for starting an imaging device and / or content prompt for entering a startup interface.

[0144] According to one or more embodiments of the present disclosure, in [Example 6], Preferably, the entire display interface corresponds to one operation control, providing an image processing method.

[0145] According to one or more embodiments of the present disclosure, in [Example 7], Preferably, when a trigger operation on the display interface is detected, activating the imaging device includes: activating the imaging device when a single tap operation on the display interface is detected; and / or activating an imaging device when a double tap operation on the display interface is detected; and / or activating the imaging device when detecting that the duration of the pressure on the display interface reaches a predetermined time threshold; and / or activating the imaging device when detecting that a touch trajectory on the display interface satisfies a predetermined trajectory; and / or activating the imaging device when detecting that a hand posture relative to the display interface corresponds to at least one predetermined posture.

[0146] According to one or more embodiments of the present disclosure, in [Example 8], Preferably, determining a target recognition result corresponding to the target object by analyzing and processing the processing target image includes: An image processing method is provided, including recognizing a target object in the processing target image based on a pre-trained image processing model, and obtaining a target recognition result corresponding to the target object.

[0147] According to one or more embodiments of the present disclosure, in [Example 9], Preferably, the target object is a traffic light, and recognizing the target object in the processing target image based on the pre-trained image processing model to obtain a target recognition result corresponding to the target object includes: inputting the processing target image into the image processing model, determining a traffic state corresponding to the traffic signal based on the image processing model, and setting the traffic state as the target recognition result; The image processing method provides that the traffic state includes a passable state or a non-passable state.

[0148] According to one or more embodiments of the present disclosure, in [Example 10], Preferably, the target object is a target vehicle or character information, and recognizing the target object in the processing target image based on the pre-trained image processing model to obtain a target recognition result corresponding to the target object includes: The image processing method includes recognizing vehicle information or text content of a target object in the processing target image based on the image processing model, and obtaining a target recognition result corresponding to the target object.

[0149] According to one or more embodiments of the present disclosure, in [Example 11], Preferably, obtaining a use object recognition result corresponding to the target object determined by at least one terminal device; updating the target recognition result based on the target recognition result used and the target recognition result output by the image processing model.

[0150] According to one or more embodiments of the present disclosure, in [Example 12], Preferably, obtaining a use object recognition result corresponding to the target object determined by at least one terminal device includes: Processing the collected target image including the target object based on at least one terminal device to obtain a target recognition result corresponding to the target object; and / or Determining the use object recognition result based on the use object recognition result corresponding to the target object received by at least one terminal device; and / or determining the target recognition result based on map data information corresponding to the target user.

[0151] According to one or more embodiments of the present disclosure, in [Example 13], Preferably, the target object corresponds to a traffic signal, the target recognition result includes a traffic status, and calling up target presentation content corresponding to the target recognition result includes: determining, under a condition that the impassable state is a state where the target presentation content is audio information and a first vibration frequency corresponding to the impassable state; and determining, under a condition that the traffic state is a passable state, that the target presentation content is audio information and a second vibration frequency corresponding to the passable state.

[0152] According to one or more embodiments of the present disclosure, in [Example 14], Presenting to the target user based on the target presentation content An image processing method is provided, including playing corresponding audio information and controlling a terminal device corresponding to the target user to vibrate at a corresponding vibration frequency.

[0153] According to one or more embodiments of the present disclosure, in [Example 15], Preferably, after starting the imaging device and entering the imaging interface, the method includes: The image processing method further includes returning to the display interface and playing back presentation information that is returned to the display interface when it is detected that a trigger operation in the imaging interface satisfies a condition for returning from the imaging interface to the display interface.

[0154] According to one or more embodiments of the present disclosure, in [Example 16], an image acquisition module configured to, when detecting a trigger operation on the display interface, activate an imaging device and capture a processing target image including a target object based on the imaging device; an image recognition module configured to analyze and process the target image to determine a target recognition result corresponding to the target object; a presentation module configured to retrieve target presentation content corresponding to the target recognition result and present it to the target user based on the target presentation content.

[0155] Also, although operations are described in a particular order, it should not be understood that the operations need not be performed in the particular order or sequence shown. In some cases, multitasking and parallel processing may be advantageous. Similarly, although the above discussion includes details of specific implementations, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of a single embodiment may also be implemented in combination in one embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either alone or in any suitable subcombination.

[0156] Although the present subject matter has been described in language specific to structural features and / or logical operations of methods, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. When a trigger operation on the display interface is detected, activating an imaging device and capturing a processing target image including a target object based on the imaging device; determining a target recognition result corresponding to the target object by analyzing and processing the target image; calling up target presentation content corresponding to the target recognition result and presenting it to the target user based on the target presentation content.

2. Obtaining current location information of the target user; The method of claim 1 , further comprising: detecting a trigger operation of the target user on the display interface if the current location information corresponds to at least one predetermined location information.

3. The method of claim 1 , further comprising: detecting a trigger operation of the target user on the display interface when detecting that current position information of a target moving object corresponds to target position information of the target user.

4. Before detecting the target user's trigger operation on the display interface, the method further comprises: launching a target application, entering a main interface of the target application, and activating the imaging device when detecting a trigger operation of the target user on the main interface; The method of claim 2 or 3, wherein the main interface corresponds to the display interface.

5. After launching the target application, the method includes: Further comprising: providing an application launch prompt to the target user, and performing a trigger operation on the display interface based on the application launch prompt; The method of claim 4 , wherein the application activation prompt includes at least one of an audio prompt or a vibration prompt, and the content of the audio prompt includes at least one of a trigger operation content for activating the imaging device or a content prompt for entering a startup interface.

6. The method of claim 1 , wherein the entire display interface corresponds to one operational control.

7. activating the imaging device when a trigger operation on the display interface is detected, activating the imaging device when a single tap operation on the display interface is detected; or activating the imaging device when a double tap operation on the display interface is detected; or activating the imaging device when detecting that the duration of the press on the display interface reaches a predetermined time threshold; or activating the imaging device when detecting that a touch trace on the display interface satisfies a predetermined trace; or activating the imaging device when detecting that a hand posture relative to the display interface corresponds to at least one predetermined posture.

8. determining a target recognition result corresponding to the target object by analyzing and processing the processing target image; The method of claim 1 , further comprising: recognizing a target object in the processing target image based on a pre-trained image processing model to obtain a target recognition result corresponding to the target object.

9. The target object is a traffic signal, and recognizing the target object in the processing target image based on the pre-trained image processing model to obtain a target recognition result corresponding to the target object includes: inputting the processing target image into the image processing model, determining a traffic state corresponding to the traffic signal based on the image processing model, and setting the traffic state as the target recognition result; The method of claim 8 , wherein the traffic state includes a passable state or a non-passable state.

10. The target object is a target vehicle or text information, and recognizing the target object in the processing target image based on the pre-trained image processing model to obtain a target recognition result corresponding to the target object is The method according to claim 8 , further comprising: recognizing vehicle information or text content of the target object in the processing target image based on the image processing model to obtain a target recognition result corresponding to the target object.

11. Obtaining a use object recognition result corresponding to the target object determined by at least one terminal device; and updating the target recognition result based on the used object recognition result and the target recognition result output by the image processing model.

12. Obtaining a use object recognition result corresponding to the target object determined by at least one terminal device includes: Processing the collected processing object image including the target object based on the at least one terminal device to obtain a usage object recognition result corresponding to the target object; or determining the use object recognition result based on the use object recognition result corresponding to the target object received by the at least one terminal device; or and determining the target recognition result based on map data information corresponding to the target user.

13. The target object corresponds to a traffic signal, and the target recognition result includes a traffic status, and calling up target presentation content corresponding to the target recognition result includes: determining, under a condition that the traffic state is an impassable state, that the target presentation content is audio information and a first vibration frequency corresponding to the impassable state; and determining, under a condition that the traffic state is a passable state, that the target presentation content is audio information and a second vibration frequency corresponding to the passable state.

14. Presenting to the target user based on the target presentation content The method of claim 13 , comprising: playing audio information corresponding to the traffic state; and controlling a terminal device corresponding to the target user to vibrate at a vibration frequency corresponding to the traffic state.

15. After starting the imaging device and entering an imaging interface, the method includes: The method of claim 1 , further comprising: returning to the display interface and playing presentation information that returns to the display interface when detecting that a trigger operation in the imaging interface satisfies a condition for returning from the imaging interface to the display interface.

16. an image acquisition module configured to, when detecting a trigger operation on the display interface, activate an imaging device and capture a processing target image including a target object based on the imaging device; an image recognition module configured to analyze and process the target image to determine a target recognition result corresponding to the target object; a presentation module configured to retrieve target presentation content corresponding to the target recognition result and present it to the target user based on the target presentation content.

17. one or more processors; a storage device configured to store one or more programs, An electronic device that, when the one or more programs are executed by the one or more processors, causes the one or more processors to implement the image processing method according to any one of claims 1 to 15.

18. A storage medium containing computer-executable instructions which, when executed by a computer processor, cause the image processing method of any one of claims 1 to 15 to be performed.

Images