Target recognition method, electronic device and storage medium
By introducing the final scaling parameter decision mechanism in electronic devices, the problem that repeated amplification of the target image caused by multiple algorithms to calculate the scaling parameters will reduce the recognition efficiency, achieving more efficient target recognition and improving user experience.
Patent Information
- Application Number
- CN202310978548.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-03
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-08-03
AI Technical Summary
In the prior art, multiple algorithms calculate the scaling parameters separately, causing the targets in the image acquisition screen to repeatedly enlarge or shrink, reduce the target recognition efficiency and affect the user experience.
A final scaling parameter decision mechanism is introduced, and the final scaling parameters are determined through the scaling parameters generated by the first algorithm and the second algorithm, and the scaling of the target image is controlled, so as to avoid repeated enlargement or reduction, and improve recognition efficiency.
It effectively avoids repeated scaling of the target image, improving recognition efficiency and user experience.
Smart Images

Figure CN117714894B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic technology, and in particular to a target recognition method, an electronic device and a storage medium. Background Art
[0002] In order to improve user experience, some applications in electronic devices such as mobile phones and tablets are usually equipped with a code scanning function. After the user triggers the code scanning function in the application, the camera on the electronic device can capture the image of the graphic code (for example, a QR code or a barcode, etc.), and by processing the graphic code image, it can identify the information carried by the graphic code pattern in the graphic code image, and then complete operations such as receiving money, making payments, downloading and opening websites.
[0003] It is understandable that in order to facilitate the recognition of the graphic code image, the graphic code pattern in the graphic code image needs to have a suitable angle and size, that is, the camera needs to capture the graphic code image at a suitable angle and distance. For example, under normal circumstances, the optical axis of the camera should be perpendicular to the graphic code posted on the outside to be scanned, and the size of the graphic code should basically cover the entire image acquisition screen, thereby completing the acquisition of high-quality graphic code images. However, in actual application environments, due to limitations of application scenarios (for example, users scan payment codes posted at toll booths in cars), users may not be able to adjust the camera to a suitable angle and distance, resulting in the captured graphic code image being unable to be recognized. At this time, the electronic device can generate a scaling parameter based on the previous frame of the graphic code image, so as to enlarge or reduce the graphic code image according to the scaling parameter in the current frame to obtain a more easily recognizable graphic code image.
[0004] However, there may be multiple algorithms in the prior art that generate multiple zoom parameters (Zoom) based on the previous frame of the graphic code image. The zoom of the graphic code image is controlled by the multiple zoom parameters, which may cause the graphic code in the image acquisition screen to be repeatedly enlarged or reduced, that is, produce a "bellows" phenomenon, reduce the efficiency of graphic code recognition, and affect the user experience.
[0005] It should be pointed out that the information disclosed in the background technology section of this application is only intended to deepen the understanding of the general background technology of this application, and should not be regarded as an admission or suggestion in any form that the information constitutes prior art already known to those skilled in the art. Summary of the invention
[0006] In view of this, the present application provides a target recognition method, an electronic device and a storage medium, so as to solve the problem in the prior art that the scaling of the target image is controlled by scaling parameters calculated by multiple algorithms respectively, which may cause the target in the image acquisition screen to be repeatedly enlarged or reduced, thereby reducing the target recognition efficiency and affecting the user experience.
[0007] In a first aspect, an embodiment of the present application provides a target recognition method, comprising:
[0008] The first algorithm and the second algorithm generate a first scaling parameter and a second scaling parameter according to the previous frame target image respectively;
[0009] Determining a final scaling parameter according to the first scaling parameter and the second scaling parameter;
[0010] Generate a current frame target image according to the final scaling parameter, wherein the scaling ratio of the current frame target image corresponds to the final scaling parameter;
[0011] Target recognition is performed using the current frame target image.
[0012] In a possible implementation, the first algorithm and the second algorithm respectively generate the first scaling parameter and the second scaling parameter according to the previous frame target image, including: within the effective time of the second algorithm, the first algorithm and the second algorithm respectively generate the first scaling parameter and the second scaling parameter according to the previous frame target image;
[0013] The determining the final scaling parameter according to the first scaling parameter and the second scaling parameter includes: determining the final scaling parameter according to the first scaling parameter and the second scaling parameter within the effective time of the second algorithm.
[0014] In a possible implementation manner, determining a final scaling parameter according to the first scaling parameter and the second scaling parameter includes:
[0015] If the first scaling parameter is greater than the second scaling parameter, determining the first scaling parameter as a final scaling parameter;
[0016] If the first scaling parameter is less than the second scaling parameter, determining the second scaling parameter as a final scaling parameter;
[0017] If the first scaling parameter is equal to the second scaling parameter, the first scaling parameter or the second scaling parameter is determined as a final scaling parameter.
[0018] In a possible implementation manner, before generating the current frame target image according to the final scaling parameter, the method further includes:
[0019] Outside the effective time of the second algorithm, the first algorithm generates a first scaling parameter according to the last frame of the target image;
[0020] Outside the effective time of the second algorithm, the first scaling parameter is determined as a final scaling parameter.
[0021] In a possible implementation manner, determining the first scaling parameter as a final scaling parameter outside the effective time of the second algorithm includes:
[0022] Outside the effective time of the second algorithm, determining whether the first scaling parameter of the current frame changes relative to the first scaling parameter of the previous frame;
[0023] If the first scaling parameter of the current frame changes relative to the first scaling parameter of the previous frame, determining the first scaling parameter of the current frame as the final scaling parameter;
[0024] If the first scaling parameter of the current frame does not change relative to the first scaling parameter of the previous frame, the second scaling parameter obtained last time is determined as the final scaling parameter.
[0025] In a possible implementation, generating a current frame target image according to the final scaling parameter includes:
[0026] Determining a lens stretching parameter according to the final zoom parameter;
[0027] The target image is captured according to the lens stretching parameters to obtain the current frame target image.
[0028] In a possible implementation, generating a current frame target image according to the final scaling parameter includes:
[0029] Capturing the target image to obtain the initial target image of the current frame;
[0030] The current frame initial target image is scaled according to the final scaling parameter to obtain the current frame target image.
[0031] In a possible implementation manner, after generating the current frame target image according to the final scaling parameter, the method further includes:
[0032] The current frame target image is displayed.
[0033] In a possible implementation manner, the performing target recognition by using the current frame target image includes:
[0034] The first algorithm and the second algorithm respectively perform target recognition through the current frame target image.
[0035] In a possible implementation manner, the first algorithm and the second algorithm respectively perform target recognition through the current frame target image, including:
[0036] During the effective time of the second algorithm, the first algorithm and the second algorithm respectively perform target recognition through the current frame target image;
[0037] Outside the effective time of the second algorithm, the first algorithm performs target recognition through the current frame target image.
[0038] In a possible implementation, the method further includes:
[0039] If the first algorithm fails to successfully identify the target image of the current frame, generating a first scaling parameter of the current frame;
[0040] If the second algorithm fails to successfully identify the target image of the current frame, a second scaling parameter of the current frame is generated.
[0041] In a possible implementation manner, the target image is a graphic code image.
[0042] In a possible implementation manner, the first algorithm is an application algorithm, and the second algorithm is an underlying algorithm.
[0043] In a second aspect, an embodiment of the present application provides an electronic device, including:
[0044] processor;
[0045] Memory;
[0046] And a computer program, wherein the computer program is stored in the memory, and the computer program includes instructions, and when the instructions are executed by the processor, the electronic device executes any one of the methods described in the first aspect.
[0047] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described in the first aspect.
[0048] In the embodiment of the present application, a decision mechanism for the final scaling parameter is introduced, that is, the final scaling parameter is determined according to the first scaling parameter and the second scaling parameter generated by the first algorithm and the second algorithm; and then the scaling of the target image is controlled according to the final scaling parameter. This can avoid repeated enlargement or reduction of the target image, improve recognition efficiency and user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0050] Figure 1 A schematic diagram of an electronic device provided in an embodiment of the present application;
[0051] Figure 2 A software structure block diagram of an electronic device provided in an embodiment of the present application;
[0052] Figure 3A A schematic diagram of a two-dimensional code provided in an embodiment of the present application;
[0053] Figure 3B A schematic diagram of a barcode provided in an embodiment of the present application;
[0054] Figure 4A A schematic diagram of a two-dimensional code image provided in an embodiment of the present application;
[0055] Figure 4B Another schematic diagram of a two-dimensional code image provided in an embodiment of the present application;
[0056] Figure 4C Another schematic diagram of a two-dimensional code image provided in an embodiment of the present application;
[0057] Figure 5 A schematic diagram of a graphic code recognition principle in the related art;
[0058] Figure 6 A schematic diagram of a graphic code recognition principle provided in an embodiment of the present application;
[0059] Figure 7 A flowchart of a target recognition method provided in an embodiment of the present application;
[0060] Figure 8 A logical framework diagram of a target recognition method provided in an embodiment of the present application;
[0061] Fig. 9 A schematic diagram of an application scenario provided for an embodiment of the present application;
[0062] Fig.10 A flowchart of another target recognition method provided in an embodiment of the present application;
[0063] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0064] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0065] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0066] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0067] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0068] See also Figure 1 , is a schematic diagram of an electronic device provided in an embodiment of the present application. Figure 1 The electronic device is described in detail by taking the mobile phone 100 as an example. Figure 1 1 shows a front view and a rear view of a mobile phone 100, wherein a front camera 111 and a screen 120 are disposed on the front side of the mobile phone 100, and a rear camera 112 is disposed on the rear side of the mobile phone 100. Image data can be collected by the configured front camera 111 and / or rear camera 112, and after the mobile phone 100 performs a series of processing on the image data, the image collected by the front camera 111 and / or rear camera 112 can be displayed on the screen 120.
[0069] Understandable, Figure 1The above is only an exemplary description and should not be used as a limitation on the scope of protection of this application. For example, different mobile phones may have different camera configuration numbers and configuration locations. In addition, in addition to mobile phones, the electronic devices involved in the embodiments of the present application may also be tablet computers, personal computers (PCs), personal digital assistants (PDAs), smart watches, netbooks, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, vehicle-mounted devices, smart cars, smart speakers, robots, smart glasses, smart TVs, etc.
[0070] See also Figure 2 , is a software structure block diagram of an electronic device provided in an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, namely, the application layer, the framework layer, the hardware abstraction layer, and the hardware layer from top to bottom.
[0071] The application layer (Application, App) may include a series of application packages. For example, the application package may include a code scanning application, that is, an application with a code scanning function.
[0072] The Framework layer (FWK) provides application programming interfaces (APIs) and programming frameworks for applications in the application layer, including some predefined functions. Figure 2 In the framework layer, the camera framework can be a camera access interface (Camera2 API). Camera2 API is a set of interfaces for accessing camera devices launched by Android. It adopts a pipeline design to make the data flow from the camera to the Surface. Camera2 API includes camera management (CameraManager) and camera device (CameraDevice). CameraManager is the management class of the Camera device. Through this class object, the Camera device information of the device can be queried to obtain the CameraDevice object. CameraDevice provides a series of fixed parameters related to the Camera device, such as basic settings and output formats.
[0073] The hardware abstraction layer (HAL) is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. For example, HAL includes the camera hardware abstraction layer (Camera HAL), which includes camera 1, camera 2, etc. It can be understood that camera 1 and camera 2 are abstract devices. In the shooting scene, HAL will create a data stream of corresponding size according to the resolution sent by the upper layer and the size of the Surface.
[0074] The hardware layer (HardWare, HW) is the hardware at the bottom layer of the operating system. For example, HW includes camera 1, camera 2, camera 3, etc. Among them, camera 1, camera 2, camera 3 can correspond to multiple cameras on the electronic device.
[0075] In order to improve user experience, some applications in electronic devices such as mobile phones and tablets are usually equipped with a code scanning function (for example, Figure 2 After the user triggers the code scanning function in the application, the camera on the electronic device can capture the graphic code image, and by processing the graphic code image, the information carried by the graphic code pattern in the graphic code image can be identified, thereby completing operations such as receiving payment, making payment, downloading and opening a website. It should be noted that the graphic code involved in the embodiment of the present application can be a QR code (such as Figure 3A As shown), barcode (as Figure 3B As shown), or other types of patterns that can be scanned and identified, the embodiments of the present application do not impose specific limitations on this.
[0076] It is understandable that in addition to the graphic code pattern, the graphic code image usually also includes the surrounding environment information surrounding the graphic code pattern. The graphic code pattern is the effective information in the graphic code image. In order to facilitate the recognition of the graphic code image, the graphic code pattern in the graphic code image needs to have a suitable angle and size, that is, the camera needs to capture the graphic code image at a suitable angle and distance. For example, under normal circumstances, the optical axis of the camera should be perpendicular to the graphic code posted on the outside and to be scanned, and the size of the graphic code should basically cover the entire image capture screen, thereby completing the capture of a high-quality graphic code image. Exemplary, in Figure 4A In the two-dimensional code image shown, the two-dimensional code pattern is not distorted, and the size of the two-dimensional code pattern basically covers the entire two-dimensional code image. It can be understood that the two-dimensional code image is relatively easy to be recognized.
[0077] However, in actual application environments, due to the limitations of the application scenario (for example, the user scans the payment code posted at the toll booth in the car), the user may not be able to adjust the camera to the appropriate angle and distance, which may cause the captured graphic code image to be unrecognizable. Specifically, when the optical axis of the camera is at a large inclination angle with the plane where the graphic code posted outside and to be scanned is located, the graphic code pattern in the captured graphic code image will be severely distorted. For example, in Figure 4B In the QR code image shown, the QR code pattern is severely distorted, which may cause the QR code image to be unrecognizable. Alternatively, when the camera is far away from the graphic code posted outside and to be scanned, the graphic code pattern in the captured graphic code image is small. Figure 4C In the QR code image shown, the QR code pattern is small, so the QR code image may not be recognizable. At this time, the electronic device can generate a scaling parameter based on the previous frame of the graphic code image, so as to enlarge or reduce the graphic code in the image acquisition screen according to the scaling parameter in the current frame, and obtain a more recognizable graphic code image. This is described in detail below with reference to the accompanying drawings.
[0078] See also Figure 5 , is a schematic diagram of a graphic code recognition principle in the related art. Figure 5 The ISP module, the barcode scanning application, and the underlying algorithm are shown in FIG. Among them, the ISP module is used to process the data collected by the front-end sensor to output image data; the barcode scanning is an application with a barcode scanning function; the underlying algorithm is an algorithm installed at the bottom layer according to the layered software architecture, such as Figure 2 The HAL layer shown.
[0079] Specifically, the ISP module includes an Image Front Engine (IFE) module, an Image Processing Engine (IPE) module, etc. The data output by the sensor will first reach the IFE module, which will perform some color correction, downsampling, de-mosaicing, and statistical 3A data processing. The IPE module is responsible for image processing tasks such as hardware noise reduction (MFNR, MFSR), image cropping, noise reduction, color processing, detail enhancement, etc. The graphic code image data collected by the sensor is sent to the code scanning application for display (display data stream); it is sent to the code scanning application through the underlying algorithm for graphic code image recognition (callback data stream). Among them, the underlying algorithm can also recognize graphic code images.
[0080] If the code scanning application cannot complete the recognition of the graphic code image, a zoom parameter (Zoom value) is returned to the ISP module to zoom in or out the next frame of the graphic code image; if the underlying algorithm cannot complete the recognition of the graphic code image, a zoom parameter (Zoom value) ISP value is also returned to zoom in or out the next frame of the graphic code image. For the sake of distinction, in the embodiment of the present application, the zoom parameter generated by the code scanning application is referred to as the "first zoom parameter"; the zoom parameter generated by the underlying algorithm is referred to as the "second zoom parameter".
[0081] In some application scenarios, the first scaling parameter and the second scaling parameter may differ greatly. If the scaling of the graphic code image is controlled according to the first scaling parameter and the second scaling parameter, the graphic code in the image acquisition screen may be repeatedly enlarged or reduced, i.e., a "bellows phenomenon" may occur, which reduces the efficiency of graphic code recognition and affects the user experience.
[0082] See also Figure 6 , is a schematic diagram of a graphic code recognition principle provided by an embodiment of the present application. Figure 6 As shown, the embodiment of the present application is Figure 5 Based on the above, a scaling parameter decision module is added. The first scaling parameter generated by the code scanning application and the second scaling parameter generated by the underlying algorithm are sent to the scaling parameter decision module respectively, and the scaling parameter decision module determines the final scaling parameter according to the first scaling parameter and the second scaling parameter, and sends the final scaling parameter to the ISP module, and the ISP module determines the scaling ratio of the graphic code image according to the final scaling parameter.
[0083] That is to say, in the embodiment of the present application, the final scaling parameter is first determined according to the first scaling parameter and the second scaling parameter generated by the code scanning application and the underlying algorithm; and then the scaling of the graphic code image is controlled according to the final scaling parameter. This can avoid repeated enlargement or reduction of the graphic code image, improve recognition efficiency and user experience. This is described in detail below in conjunction with a specific implementation method.
[0084] See also Figure 7 , which is a flow chart of a target recognition method provided in an embodiment of the present application; see Figure 8 , is a logical framework diagram of a target recognition method provided in an embodiment of the present application. The method can be applied to Figure 1 and Figure 2 The electronic equipment shown, such as Figure 7 Combined with Figure 8 As shown, it mainly includes the following steps.
[0085] Step S701: The first algorithm and the second algorithm generate a first scaling parameter and a second scaling parameter respectively according to a target image of a previous frame.
[0086] The "target" involved in the embodiment of the present application may be the graphic code described above, such as a QR code, a barcode, etc. Of course, in some object recognition application scenarios, the "target" may also be an object to be recognized, such as an animal, a plant, a toy, a person, etc., and the embodiment of the present application does not specifically limit this.
[0087] In a possible implementation, the first algorithm may be an application algorithm, that is, an algorithm in an application program (also referred to as an "application") installed in an electronic device; the second algorithm may be a bottom-level algorithm, that is, an algorithm installed in the bottom layer of the electronic device, such as an algorithm installed in a Figure 2 The HAL layer shown. In a specific implementation, an application is usually a program installed by a user in an electronic device according to his or her own needs. Accordingly, the application algorithm is the algorithm carried by the application installed by the user; the underlying algorithm is usually the algorithm that comes with the operating system of the electronic device. It can be understood that when the target to be identified is a graphic code, the application is a code scanning application (such as Figure 2 ) or an application with a code scanning function.
[0088] It should be noted that in some possible application scenarios, the first algorithm and the second algorithm may both be application algorithms, or both be underlying algorithms, and the present application embodiment does not specifically limit this. For ease of understanding, the following takes the first algorithm as an application algorithm and the second algorithm as an underlying algorithm as an example to exemplify the technical solution provided in the present application embodiment.
[0089] Specifically, when the user needs to identify a target, he can first aim the camera of the electronic device at the target to be identified, then open the application and trigger the target identification instruction in the application; after the application receives the target identification instruction triggered by the user, it sends a target identification request to the preprocessing module; after the preprocessing module preprocesses the target identification request, it sends the preprocessed target identification request to the ISP module, and the ISP module outputs the first frame of the target image according to the target identification request. It should be pointed out that for the first frame of the target image, since the application algorithm and the underlying algorithm have not yet generated the first scaling parameter and the second scaling parameter, the first frame of the target image can be generated based on the scaling parameters sent by the application.
[0090] After the first frame of the target image is generated, the application algorithm and the underlying algorithm respectively identify the first frame of the target image. If the application algorithm cannot complete the identification of the first frame of the target image, the application algorithm generates the first scaling parameter of the first frame according to the first frame of the target image, and the first scaling parameter of the first frame is used to indicate the scaling ratio of the next frame of the target image (the second frame of the target image); if the underlying algorithm cannot complete the identification of the first frame of the target image, the underlying algorithm generates the second scaling parameter of the first frame according to the first frame of the target image, and the second scaling parameter of the first frame is used to indicate the scaling ratio of the next frame of the target image (the second frame of the target image). Similarly, if the application algorithm cannot complete the identification of the second frame of the target image, the application algorithm generates the first scaling parameter of the second frame according to the second frame of the target image, and the first scaling parameter of the second frame is used to indicate the scaling ratio of the next frame of the target image (the third frame of the target image); if the underlying algorithm cannot complete the identification of the second frame of the target image, the underlying algorithm generates the second scaling parameter of the second frame according to the second frame of the target image, and the second scaling parameter of the second frame is used to indicate the scaling ratio of the next frame of the target image (the third frame of the target image), and so on. The embodiments of the present application will not be repeated here.
[0091] In practical applications, the target recognition effect can be improved by using the underlying algorithm to participate in target recognition. However, a target recognition process is usually not too long, for example, 2-5 seconds. If the underlying algorithm is turned on for a long time, the system power consumption will increase.
[0092] In response to the above problem, in one possible implementation, the effective time of the underlying algorithm is set. During the effective time of the underlying algorithm, the underlying algorithm is turned on and participates in target recognition; after the effective time of the underlying algorithm is exceeded, the underlying algorithm is turned off and the underlying algorithm no longer participates in target recognition. It can be understood that when the underlying algorithm is turned on, that is, within the effective time of the underlying algorithm, the first scaling parameter and the second scaling parameter can be generated respectively. When the underlying algorithm is turned off, that is, outside the effective time of the underlying algorithm, the first scaling parameter is only generated by the application algorithm.
[0093] Since the second scaling parameter can be generated only after the underlying algorithm is enabled, step S701 corresponds to the case where the underlying algorithm is enabled, that is, the application algorithm and the underlying algorithm generate the first scaling parameter and the second scaling parameter according to the previous frame target image, respectively, including: within the effective time of the underlying algorithm, the application algorithm and the underlying algorithm generate the first scaling parameter and the second scaling parameter according to the previous frame target image, respectively. In addition, outside the effective time of the underlying algorithm, the application algorithm generates the first scaling parameter according to the previous frame target image.
[0094] In a possible implementation, the underlying algorithm can be effective within a certain period of time after the user opens the application. That is, when the user opens the application, the underlying algorithm is immediately turned on; after exceeding the set time period, the underlying algorithm is turned off. Those skilled in the art can set the specific length of the time period according to actual needs, for example, 5-10 seconds, etc., and the embodiments of the present application are not limited to this.
[0095] Step S702: Determine a final scaling parameter according to the first scaling parameter and the second scaling parameter.
[0096] Please continue reading Figure 8 In an embodiment of the present application, the application algorithm and the underlying algorithm respectively send the generated first scaling parameter and the second scaling parameter to the preprocessing module, and the scaling parameter decision module in the preprocessing module determines the final scaling parameter based on the first scaling parameter and the second scaling parameter.
[0097] As described above, in some possible implementations, in order to save system power consumption, the effective time of the underlying algorithm can be set. Accordingly, the final scaling parameter is determined according to the first scaling parameter and the second scaling parameter, specifically including: within the effective time of the underlying algorithm, the final scaling parameter is determined according to the first scaling parameter and the second scaling parameter. In addition, outside the effective time of the underlying algorithm, the first scaling parameter is determined as the final scaling parameter.
[0098] In a possible implementation, during the effective time of the underlying algorithm, the final scaling parameter is determined according to the first scaling parameter and the second scaling parameter, specifically including: if the first scaling parameter is greater than the second scaling parameter, the first scaling parameter is determined to be the final scaling parameter; if the first scaling parameter is less than the second scaling parameter, the second scaling parameter is determined to be the final scaling parameter; if the first scaling parameter is equal to the second scaling parameter, the first scaling parameter or the second scaling parameter is determined to be the final scaling parameter. In other words, the larger value of the first scaling parameter and the second scaling parameter is used as the final scaling parameter. Exemplarily, if the first scaling parameter is 1.5 and the second scaling parameter is 2, the second scaling parameter is used as the final scaling parameter; or, if the first scaling parameter is 3 and the second scaling parameter is 2, the first scaling parameter is used as the final scaling parameter.
[0099] In the embodiment of the present application, the larger value of the first scaling parameter and the second scaling parameter is used as the final scaling parameter, so that the target image can be quickly enlarged, thereby improving the target recognition efficiency. On the contrary, if the smaller value of the first scaling parameter and the second scaling parameter is used as the final scaling parameter, it may result in the need to scale the target image more times before the target image can be recognized, thereby reducing the target recognition efficiency.
[0100] In a possible implementation, outside the effective time of the underlying algorithm, determining the first scaling parameter as the final scaling parameter specifically includes: outside the effective time of the underlying algorithm, judging whether the first scaling parameter of the current frame has changed relative to the first scaling parameter of the previous frame; if the first scaling parameter of the current frame has changed relative to the first scaling parameter of the previous frame, determining the first scaling parameter of the current frame as the final scaling parameter; if the first scaling parameter of the current frame has not changed relative to the first scaling parameter of the previous frame, determining the second scaling parameter obtained for the last time as the final scaling parameter.
[0101] Exemplarily, the current frame is the kth frame, and the previous frame is the k-1th frame, k>1. If the first scaling parameter of the kth frame changes relative to the first scaling parameter of the k-1th frame, the first scaling parameter of the kth frame is determined to be the final scaling parameter. The first scaling parameter of the kth frame changes relative to the first scaling parameter of the k-1th frame. On the one hand, it may be due to the user manually performing a scaling operation; on the other hand, it may be due to the application algorithm further adjusting the first scaling parameter. At this time, using the first scaling parameter of the kth frame as the final scaling parameter can ensure a quick response to the user's manual scaling operation; and ensure that the latest first scaling parameter is used to scale the next frame of the target image.
[0102] In addition, if the first zoom parameter of the kth frame has not changed relative to the first zoom parameter of the k-1th frame, it means that there is no manual zoom operation by the user; and the application algorithm has not further adjusted the first zoom parameter. Usually, the second zoom parameter calculated by the underlying algorithm is more accurate. In this case, using the second zoom parameter obtained for the last time as the final zoom parameter can improve the target recognition effect. The "last obtained second zoom parameter" is the second zoom parameter obtained for the last time before the underlying algorithm is closed.
[0103] It should be noted that the preprocessing module involved in the embodiment of the present application can perform other processing in addition to deciding the final scaling parameters, for example, adjusting the scaling parameters to coordinate data so that the ISP module completes the scaling of the target image based on the coordinate data.
[0104] Step S703: generating a current frame target image according to the final scaling parameter, wherein the scaling ratio of the current frame target image corresponds to the final scaling parameter.
[0105] After determining the final scaling parameters, the embodiment of the present application can scale the target image according to the final scaling parameters to generate the current frame target image. It can be understood that the previous frame target image in step S701 refers to the previous frame of the current frame target image. Exemplarily, if the current frame is the kth frame, the previous frame is the k-1th frame, k>1.
[0106] See also Fig. 9 , is a schematic diagram of an application scenario provided by an embodiment of the present application. Fig. 9 As shown, in the obtained k-1 frame of the two-dimensional code image, the two-dimensional code pattern is small, resulting in the k-1 frame of the two-dimensional code image not being recognized. According to the above method, the final scaling parameter is generated, and the two-dimensional code image is scaled based on the final scaling parameter to generate the k-th frame of the two-dimensional code image. It can be understood that the k-th frame of the two-dimensional code image after enlargement is easier to be recognized.
[0107] In specific implementation, the zooming of the target image can be achieved based on two methods: optical zoom and digital zoom. Optical zoom refers to zooming achieved through physical optical means, that is, adjusting the distance between the lenses of the hardware; digital zoom is achieved through software means, that is, enlarging or reducing the target image through image processing. The following are explanations of each method.
[0108] When scaling the target image based on optical zoom, after obtaining the final scaling parameters, the camera lens stretching parameters can be determined according to the final scaling parameters; the camera lens is stretched (focused) according to the lens stretching parameters, and then the target image is captured to obtain the current frame target image.
[0109] When scaling the target image based on digital zoom, after obtaining the final scaling parameters, the target image is captured to obtain the initial target image of the current frame, and then the initial target image of the current frame is scaled according to the final scaling parameters to obtain the target image of the current frame. In other words, the image captured by the camera does not change, but the scaling of the target image is achieved through image processing.
[0110] Step S704: performing target recognition through the current frame target image.
[0111] In an embodiment of the present application, after obtaining the current frame target image, target recognition is performed through the current frame target image. If target recognition can be achieved, corresponding processing is performed according to the target recognition result (for example, collecting money, paying, downloading, and opening a website are performed according to the graphic code recognition result), and the recognition process ends; if target recognition cannot be achieved, the first scaling parameter and the second scaling parameter are regenerated according to the current frame target image, which is equivalent to returning to step S701. The specific process can be found in the above description, and the embodiment of the present application will not be repeated here.
[0112] In some possible implementations, the application algorithm and the underlying algorithm may respectively perform target recognition through the current frame target image. If the application algorithm fails to successfully recognize the target image through the current frame, a first scaling parameter of the current frame is generated; if the underlying algorithm fails to successfully recognize the target image through the current frame, a second scaling parameter of the current frame is generated.
[0113] As described above, in some possible implementations, the effective time of the underlying algorithm is limited. It is understandable that during the effective time of the underlying algorithm, since the underlying algorithm is turned on, the application algorithm and the underlying algorithm can respectively perform target recognition through the current frame target image; outside the effective time of the underlying algorithm, since the underlying algorithm is turned off, the underlying algorithm cannot be used for target recognition at this time, and therefore, the target image is only recognized by the application algorithm.
[0114] In addition, after the current frame target image is obtained, the current frame target image can be displayed on the display screen so that the user can understand the scaling status of the target image in real time, thereby improving the user experience.
[0115] In the embodiment of the present application, a decision mechanism for the final scaling parameter is introduced, that is, the final scaling parameter is determined according to the first scaling parameter and the second scaling parameter generated by the first algorithm and the second algorithm; and then the scaling of the target image is controlled according to the final scaling parameter. This can avoid repeated enlargement or reduction of the target image, improve recognition efficiency and user experience.
[0116] For ease of understanding, the technical solution provided in the embodiment of the present application is exemplarily described below in conjunction with a specific implementation method.
[0117] See also Fig.10 , is a flow chart of another target recognition method provided in an embodiment of the present application. Fig.10 As shown, it mainly includes the following steps.
[0118] Step S1001: determine whether it is within the effective time of the second algorithm, if the determination result is yes, proceed to step S1002, otherwise, proceed to step S1006;
[0119] Step S1002: start the second algorithm;
[0120] Step S1003: determine whether the first scaling parameter is less than the second scaling parameter, if the determination result is yes, proceed to step S1004, otherwise, proceed to step S1005;
[0121] Step S1004: Determine the second scaling parameter as the final scaling parameter;
[0122] Step S1005: Determine the first scaling parameter as the final scaling parameter;
[0123] Step S1006: close the second algorithm;
[0124] Step S1007: determining whether the first scaling parameter of the current frame has changed relative to the first scaling parameter of the previous frame, if the determination result is yes, proceeding to step S1008, otherwise, proceeding to step S1009;
[0125] Step S1008: Determine the first scaling parameter of the current frame as the final scaling parameter;
[0126] Step S1009: determining the second scaling parameter obtained last time as the final scaling parameter;
[0127] Step S1010: generating a current frame target image according to the final scaling parameters;
[0128] Step S1011: performing target recognition through the current frame target image.
[0129] It should be pointed out that the specific details involved in the embodiments of the present application can be found in the description of the above embodiments, and for the sake of brevity, they will not be repeated here.
[0130] Corresponding to the above-mentioned embodiments, the present application also provides an electronic device, which includes a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute some or all of the steps in the above-mentioned method embodiments.
[0131] See also Fig.11 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.11 As shown, the electronic device 1100 may include a processor 1110, an external memory interface 1120, an internal memory 1121, a universal serial bus (USB) interface 1130, a charging management module 1140, a power management module 1141, a battery 1142, an antenna 1, an antenna 2, a mobile communication module 1150, a wireless communication module 1160, an audio module 1170, a speaker 1170A, a receiver 1170B, a microphone 1170C, an earphone interface 1170D, a sensor module 1180, a button 1190, a motor 1191, an indicator 1192, a camera 1193, a display screen 1194, and a subscriber identification module (SIM) card interface 1195, etc. The sensor module 1180 may include a pressure sensor 1180A, a gyroscope sensor 1180B, an air pressure sensor 1180C, a magnetic sensor 1180D, an acceleration sensor 1180E, a distance sensor 1180F, a proximity light sensor 1180G, a fingerprint sensor 1180H, a temperature sensor 1180J, a touch sensor 1180K, an ambient light sensor 1180L, a bone conduction sensor 1180M, and the like.
[0132] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 1100. In other embodiments of the present application, the electronic device 1100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0133] The processor 1110 may include one or more processing units, for example, the processor 1110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0134] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.
[0135] A memory may also be provided in the processor 1110 for storing instructions and data. In some embodiments, the memory in the processor 1110 is a cache memory. The memory may store instructions or data that the processor 1110 has just used or cyclically used. If the processor 1110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 1110, and thus improves the efficiency of the system.
[0136] In some embodiments, the processor 1110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0137] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 1110 may include multiple groups of I2C buses. The processor 1110 may be coupled to the touch sensor 1180K, the charger, the flash, the camera 1193, etc. through different I2C bus interfaces.
[0138] The I2S interface can be used for audio communication. In some embodiments, the processor 1110 can include multiple I2S buses. The processor 1110 can be coupled to the audio module 1170 via the I2S bus to achieve communication between the processor 1110 and the audio module 1170.
[0139] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 1170 and the wireless communication module 1160 can be coupled via a PCM bus interface.
[0140] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 1110 and the wireless communication module 1160.
[0141] The MIPI interface can be used to connect the processor 1110 with peripheral devices such as the display screen 1194 and the camera 1193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 1110 and the camera 1193 communicate via the CSI interface to implement the shooting function of the electronic device 1100. The processor 1110 and the display screen 1194 communicate via the DSI interface to implement the display function of the electronic device 1100.
[0142] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 1110 with the camera 1193, the display 1194, the wireless communication module 1160, the audio module 1170, the sensor module 1180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.
[0143] The USB interface 1130 is an interface that complies with USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 1130 may be used to connect a charger to charge the electronic device 1100, and may also be used to transmit data between the electronic device 1100 and a peripheral device.
[0144] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 1100. In other embodiments of the present application, the electronic device 1100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0145] The charging management module 1140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 1140 may receive charging input from a wired charger through the USB interface 1130. In some wireless charging embodiments, the charging management module 1140 may receive wireless charging input through a wireless charging coil of the electronic device 1100. While the charging management module 1140 is charging the battery 1142, it may also power the terminal through the power management module 1141.
[0146] The power management module 1141 is used to connect the battery 1142, the charging management module 1140 and the processor 1110. The power management module 1141 receives input from the battery 1142 and / or the charging management module 1140, and supplies power to the processor 1110, the internal memory 1121, the display screen 1194, the camera 1193, and the wireless communication module 1160. The power management module 1141 can also be used to monitor parameters such as battery capacity, battery cycle number, and battery health status (leakage, impedance).
[0147] The wireless communication function of the electronic device 1100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 1150, the wireless communication module 1160, the modem processor and the baseband processor.
[0148] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 1100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0149] The mobile communication module 1150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 1100. The mobile communication module 1150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 1150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 1150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 1150 can be set in the processor 1110. In some embodiments, at least some of the functional modules of the mobile communication module 1150 can be set in the same device as at least some of the modules of the processor 1110.
[0150] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 1170A, a receiver 1170B, etc.), or displays an image or video through a display screen 1194.
[0151] The wireless communication module 1160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 1100. The wireless communication module 1160 can be one or more devices integrating at least one communication processing module. The wireless communication module 1160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 1110. The wireless communication module 1160 can also receive the signal to be sent from the processor 1110, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0152] In some embodiments, the antenna 1 of the electronic device 1100 is coupled to the mobile communication module 1150, and the antenna 2 is coupled to the wireless communication module 1160, so that the electronic device 1100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).
[0153] The electronic device 1100 implements the display function through a GPU, a display screen 1194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 1194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 1110 may include one or more GPUs that execute program instructions to generate or change display information.
[0154] The display screen 1194 is used to display images, videos, etc. The display screen 1194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 1100 may include 1 or N display screens 1194, where N is a positive integer greater than 1.
[0155] The electronic device 1100 can realize the shooting function through an ISP, a camera 1193, a video codec, a GPU, a display screen 1194, and an application processor.
[0156] The ISP is used to process the data fed back by the camera 1193. For example, when taking a photo, the shutter is opened, and light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image.
[0157] The camera 1193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. In some embodiments, the electronic device 1100 may include 1 or N cameras 1193, where N is a positive integer greater than 1.
[0158] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 1100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
[0159] Video codecs are used to compress or decompress digital videos. The electronic device 1100 may support one or more video codecs. Thus, the electronic device 1100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0160] NPU is a neural network (NN) computing processor, which can quickly process input information and continuously self-learn by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain. NPU can realize applications such as intelligent cognition of electronic device 1100.
[0161] The external memory interface 1120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 1100. The external memory card communicates with the processor 1110 through the external memory interface 1120 to implement a data storage function, such as storing music, video and other files in the external memory card.
[0162] The internal memory 1121 can be used to store computer executable program codes, which include instructions. The internal memory 1121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 1100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 1121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 1110 executes various functional applications and data processing of the electronic device 1100 by running instructions stored in the internal memory 1121 and / or instructions stored in a memory provided in the processor.
[0163] The electronic device 1100 can implement audio functions such as music playing and recording through the audio module 1170, the speaker 1170A, the receiver 1170B, the microphone 1170C, the headphone jack 1170D, and the application processor.
[0164] The audio module 1170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 1170 can also be used to encode and decode audio signals.
[0165] The speaker 1170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 1100 can listen to music or listen to a hands-free call through the speaker 1170A.
[0166] The receiver 1170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 1100 receives a call or voice message, the voice can be received by placing the receiver 1170B close to the human ear.
[0167] Microphone 1170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by approaching the microphone 1170C with his mouth to input the sound signal into the microphone 1170C. The electronic device 1100 can be provided with at least one microphone 1170C.
[0168] The earphone interface 1170D is used to connect a wired earphone and can be a USB interface 1130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.
[0169] The pressure sensor 1180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor 1180A can be set on the display screen 1194. There are many types of pressure sensors 1180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 1180A, the capacitance between the electrodes changes. The electronic device 1100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 1194, the electronic device 1100 detects the intensity of the touch operation based on the pressure sensor 1180A. The electronic device 1100 can also calculate the position of the touch based on the detection signal of the pressure sensor 1180A.
[0170] The gyro sensor 1180B can be used to determine the motion posture of the electronic device 1100. In some embodiments, the angular velocity of the electronic device 1100 around three axes (ie, x, y, and z axes) can be determined by the gyro sensor 1180B. The gyro sensor 1180B can be used for anti-shake shooting.
[0171] The air pressure sensor 1180C is used to measure air pressure. In some embodiments, the electronic device 1100 calculates the altitude through the air pressure value measured by the air pressure sensor 1180C to assist positioning and navigation.
[0172] The magnetic sensor 1180D includes a Hall sensor, and the electronic device 1100 can use the magnetic sensor 1180D to detect the opening and closing of the flip leather case.
[0173] The acceleration sensor 1180E can detect the magnitude of the acceleration of the electronic device 1100 in all directions (generally three axes). When the electronic device 1100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the terminal posture and is applied to applications such as horizontal and vertical screen switching and pedometers.
[0174] The distance sensor 1180F is used to measure the distance. The electronic device 1100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 1100 can use the distance sensor 1180F to measure the distance to achieve fast focusing.
[0175] The proximity light sensor 1180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 1100 emits infrared light outward through the light emitting diode. The electronic device 1100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 1100.
[0176] The ambient light sensor 1180L is used to sense the brightness of the ambient light. The electronic device 1100 can adaptively adjust the brightness of the display screen 1194 according to the sensed brightness of the ambient light.
[0177] The fingerprint sensor 1180H is used to collect fingerprints. The electronic device 1100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.
[0178] The temperature sensor 1180J is used to detect temperature. In some embodiments, the electronic device 1100 uses the temperature detected by the temperature sensor 1180J to execute a temperature processing strategy.
[0179] The touch sensor 1180K is also called a "touch control device". The touch sensor 1180K can be set on the display screen 1194. The touch sensor 1180K and the display screen 1194 form a touch screen, also called a "touch control screen". The touch sensor 1180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 1194. In other embodiments, the touch sensor 1180K can also be set on the surface of the electronic device 1100, which is different from the position of the display screen 1194.
[0180] The bone conduction sensor 1180M can obtain vibration signals. In some embodiments, the bone conduction sensor 1180M can obtain vibration signals of vibrating bones of the human body. The bone conduction sensor 1180M can also contact the human body's pulse to receive blood pressure beating signals.
[0181] The key 1190 includes a power key, a volume key, etc. The key 1190 may be a mechanical key or a touch key. The electronic device 1100 may receive key input and generate key signal input related to user settings and function control of the electronic device 1100.
[0182] Motor 1191 can generate vibration prompts. Motor 1191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
[0183] Indicator 1192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0184] The SIM card interface 1195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 1100 by inserting it into the SIM card interface 1195 or pulling it out from the SIM card interface 1195. The electronic device 1100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 1195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 1195 at the same time. The types of the multiple cards can be the same or different. The electronic device 1100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 1100 uses an eSIM, i.e., an embedded SIM card.
[0185] In a specific implementation, the present application also provides a computer storage medium, wherein the computer storage medium may store a program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to perform some or all of the steps in the above embodiment. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).
[0186] In a specific implementation, an embodiment of the present application further provides a computer program product, which includes executable instructions. When the executable instructions are executed on a computer, the computer executes some or all of the steps in the above method embodiment.
[0187] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.
[0188] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.
[0189] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0190] In several embodiments provided by the present invention, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.
[0191] The above is only a specific embodiment of the present invention. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A target recognition method, characterized in that: include: The first algorithm and the second algorithm generate a first scaling parameter and a second scaling parameter respectively according to the image data of the target image of the previous frame, wherein the first scaling parameter and the second scaling parameter are respectively used to indicate the scaling ratio of the target image of the current frame; Determine a final scaling parameter according to the first scaling parameter and the second scaling parameter, the final scaling parameter being the first scaling parameter or the second scaling parameter; Generate a current frame target image according to the final scaling parameter, wherein the scaling ratio of the current frame target image corresponds to the final scaling parameter; Target recognition is performed using the current frame target image.
2. The method according to claim 1, characterized in that The first algorithm and the second algorithm generate the first scaling parameter and the second scaling parameter respectively according to the previous frame target image, including: within the effective time of the second algorithm, the first algorithm and the second algorithm generate the first scaling parameter and the second scaling parameter respectively according to the previous frame target image; The determining the final scaling parameter according to the first scaling parameter and the second scaling parameter includes: determining the final scaling parameter according to the first scaling parameter and the second scaling parameter within the effective time of the second algorithm.
3. The method according to claim 1 or 2, characterized in that: The determining a final scaling parameter according to the first scaling parameter and the second scaling parameter comprises: If the first scaling parameter is greater than the second scaling parameter, determining the first scaling parameter as a final scaling parameter; If the first scaling parameter is less than the second scaling parameter, determining the second scaling parameter as a final scaling parameter; If the first scaling parameter is equal to the second scaling parameter, the first scaling parameter or the second scaling parameter is determined as a final scaling parameter.
4. The method according to claim 2, characterized in that: Before generating the current frame target image according to the final scaling parameters, the method further includes: Outside the effective time of the second algorithm, the first algorithm generates a first scaling parameter according to the last frame of the target image; Outside the effective time of the second algorithm, the first scaling parameter is determined as a final scaling parameter.
5. The method according to claim 4, characterized in that The determining, outside the effective time of the second algorithm, that the first scaling parameter is a final scaling parameter includes: Outside the effective time of the second algorithm, determining whether the first scaling parameter of the current frame changes relative to the first scaling parameter of the previous frame; If the first scaling parameter of the current frame changes relative to the first scaling parameter of the previous frame, determining the first scaling parameter of the current frame as the final scaling parameter; If the first scaling parameter of the current frame does not change relative to the first scaling parameter of the previous frame, the second scaling parameter obtained last time is determined as the final scaling parameter.
6. The method according to claim 1, characterized in that The step of generating a current frame target image according to the final scaling parameter comprises: Determining a lens stretching parameter according to the final zoom parameter; The target image is captured according to the lens stretching parameters to obtain the current frame target image.
7. The method according to claim 1, characterized in that The step of generating a current frame target image according to the final scaling parameter comprises: Capturing the target image to obtain the initial target image of the current frame; The current frame initial target image is scaled according to the final scaling parameter to obtain the current frame target image.
8. The method according to claim 1, characterized in that After generating the current frame target image according to the final scaling parameter, the method further includes: The current frame target image is displayed.
9. The method according to claim 1, characterized in that: The performing target recognition through the current frame target image includes: The first algorithm and the second algorithm respectively perform target recognition through the current frame target image.
10. The method according to claim 9, characterized in that The first algorithm and the second algorithm respectively perform target recognition through the current frame target image, including: During the effective time of the second algorithm, the first algorithm and the second algorithm respectively perform target recognition through the current frame target image; Outside the effective time of the second algorithm, the first algorithm performs target recognition through the current frame target image.
11. The method according to claim 9, characterized in that Also includes: If the first algorithm fails to successfully identify the target image of the current frame, generating a first scaling parameter of the current frame; If the second algorithm fails to successfully identify the target image of the current frame, a second scaling parameter of the current frame is generated.
12. The method according to claim 1, characterized in that The target image is a graphic code image.
13. The method according to claim 1, characterized in that The first algorithm is an application algorithm, and the second algorithm is an underlying algorithm.
14. An electronic device, characterized in that: include: processor; Memory; And a computer program, wherein the computer program is stored in the memory, and the computer program includes instructions, and when the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 13.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 13.
Citation Information
Patent Citations
Object recognition method and device, storage medium and electronic device
CN113780286A
