Image processing method and device, medium, electronic equipment and vehicle

Abstract

The present disclosure belongs to the technical field of image processing, and relates to an image processing method, device, medium, electronic equipment and vehicle. The method comprises: acquiring a to-be-identified video frame, and determining a first picture feature of the to-be-identified video frame, the first picture feature representing picture quality and aesthetic degree of the to-be-identified video frame; and in response to the first picture feature satisfying a preset visual condition, triggering an image shooting operation. The present disclosure triggers the image shooting operation according to the first picture feature of the to-be-identified video frame, provides a mode of active discrimination and real-time shooting, realizes the effect of shooting and screening at the same time, solves the problem of excessive invalid data in the post-screening scheme, reduces the occurrence of high time cost and storage resource waste, improves the real-time performance and accuracy of image shooting, the overall process does not require user waiting and operation, improves the intelligent degree and automation degree of the intelligent cockpit, and optimizes the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, an image processing apparatus, a computer-readable storage medium, an electronic device, and a vehicle. Background Technology

[0002] The car's imaging equipment continuously records footage while driving, and then the footage is filtered to retain the best content. However, this method is too passive and ineffective. Summary of the Invention

[0003] To overcome the problems existing in the related technologies, this disclosure provides an image processing method, an image processing apparatus, a computer-readable storage medium, an electronic device, and a vehicle.

[0004] According to a first aspect of the present disclosure, an image processing method is provided, comprising: The video frame to be identified is acquired, and a first image feature of the video frame to be identified is determined. The first image feature characterizes the image quality and aesthetics of the video frame to be identified. In response to the first image feature satisfying the preset visual conditions, an image capture operation is triggered.

[0005] In some possible implementations, determining the first frame feature of the video frame to be identified includes: The video frames to be identified are input into a preset first recognition model to obtain a first image feature. The first image feature includes a first quality feature and a first aesthetic feature. The first quality feature represents the quality level of the two types of video frames to be identified, and the first aesthetic feature represents the aesthetics of the two types of video frames to be identified.

[0006] In some possible implementations, the step of triggering an image capture operation in response to the first image feature satisfying a preset visual condition includes: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetics, an image capture operation is triggered to obtain a captured image, which is then used to trigger an image caching operation.

[0007] In some possible implementations, the response to the method further includes: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetic appeal, the video frame to be identified is input into a preset second recognition model to obtain a second image feature. The second image feature includes a second quality feature and a second aesthetic feature. The second quality feature represents the quality level of at least three types of the video frame to be identified, and the second aesthetic feature represents the aesthetic appeal of at least three types of the video frame to be identified.

[0008] In some possible implementations, the second identification model is deployed on a server.

[0009] In some possible implementations, the method further includes: In response to the second quality feature being the target quality level and the second aesthetic feature being the target aesthetics, an image storage operation is triggered on the cached captured image.

[0010] In some possible implementations, the method further includes: In response to a mismatch between the second quality feature and the target quality level, or a mismatch between the second aesthetic feature and the target aesthetics, an image deletion operation is triggered on the cached captured image.

[0011] In some possible implementations, the first recognition model is a lightweight recognition model, and the recognition accuracy of the second recognition model is higher than that of the first recognition model.

[0012] In some possible implementations, the method further includes: Acquire video frame samples and determine a first feature sample or a second feature sample of the video frame samples; The first feature sample or the second feature sample of the video frame sample is verified to obtain a verification result, and the target sample is determined in the video frame sample based on the verification result; The first recognition model is obtained by training the target sample and the first feature sample, or the second recognition model is obtained by training the target sample and the second feature sample.

[0013] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising: The feature determination module is configured to acquire a video frame to be identified and determine a first image feature of the video frame to be identified, wherein the first image feature characterizes the image quality and aesthetics of the video frame to be identified. The image capture module is configured to trigger an image capture operation in response to the first image feature meeting a preset visual condition.

[0014] In some possible implementations, the feature determination module is configured to: The video frames to be identified are input into a preset first recognition model to obtain a first image feature. The first image feature includes a first quality feature and a first aesthetic feature. The first quality feature represents the quality level of the two types of video frames to be identified, and the first aesthetic feature represents the aesthetics of the two types of video frames to be identified.

[0015] In some possible implementations, the image capturing module is configured to: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetics, an image capture operation is triggered to obtain a captured image, which is then used to trigger an image caching operation.

[0016] According to a third aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the image processing method provided in any of the first aspects of the present disclosure.

[0017] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the image processing method provided in any of the first aspects of this disclosure.

[0018] According to a fifth aspect of the present disclosure, a vehicle is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute executable instructions stored in the memory to implement the steps of the image processing method provided in any of the first aspects of this disclosure.

[0019] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: triggering image capture operation based on the first screen features of the video frame to be identified provides an active discrimination and real-time capture method, achieving the effect of capturing and filtering simultaneously. This solves the problem of excessive invalid data in post-screening schemes, reduces the occurrence of excessive time costs and storage resource waste, improves the real-time performance and accuracy of image capture, and the entire process does not require user waiting or operation, thus optimizing the user experience.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0022] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0023] Figure 2This is a flowchart illustrating an image processing method in an application scenario according to an exemplary embodiment.

[0024] Figure 3 This is a flowchart illustrating a method for training a model in an application scenario according to an exemplary embodiment.

[0025] Figure 4 This is a flowchart illustrating a method for training a cloud model according to an exemplary embodiment.

[0026] Figure 5 This is a flowchart illustrating a method for training an edge model according to an exemplary embodiment.

[0027] Figure 6 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0028] Figure 7 This is a block diagram illustrating a vehicle according to an exemplary embodiment.

[0029] Figure 8 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0030] Figure 9 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0032] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0033] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment, such as... Figure 1 As shown, the image processing method includes the following steps: In step S110, the video frame to be identified is acquired, and the first image feature of the video frame to be identified is determined. The first image feature characterizes the image quality and aesthetics of the video frame to be identified.

[0034] In step S120, in response to the first image feature satisfying the preset visual conditions, an image capture operation is triggered.

[0035] In the exemplary embodiments of this disclosure, an image capture operation is triggered based on the first frame features of the video frame to be identified, providing an active discrimination and real-time capture method. This achieves the effect of capturing and filtering simultaneously, solving the problem of excessive invalid data in post-screening schemes, reducing excessive time costs and storage resource waste, improving the real-time performance and accuracy of image capture, and eliminating the need for user waiting and operation throughout the entire process, thus optimizing the user experience.

[0036] The following section provides a detailed explanation of each step in the image processing method.

[0037] In step S110, the video frame to be identified is acquired, and the first image feature of the video frame to be identified is determined. The first image feature characterizes the image quality and aesthetics of the video frame to be identified.

[0038] In an exemplary embodiment of this disclosure, the video frame to be identified may be captured by a camera deployed on a vehicle, or by any image or video capture device, and this exemplary embodiment does not impose any special limitations on this.

[0039] In an optional embodiment, determining the first frame features of the video frame to be identified includes: The video frames to be identified are input into a preset first recognition model to obtain the first image features. The first image features include the first quality features and the first aesthetic features. The first quality features represent the quality level of the two types of video frames to be identified, and the first aesthetic features represent the aesthetics of the two types of video frames to be identified.

[0040] The preset first recognition model can be a model deployed on end-side devices such as vehicles, used to identify and classify the first image features of the video frame to be recognized.

[0041] Inputting a single frame of the video to be identified into a preset first recognition model can output a first quality feature and a first aesthetic feature of the image. The first quality feature can include two quality levels: a target quality level and a first quality level. The target quality level is higher than the first quality level; for example, the target quality level represents clarity, while the first quality level represents blurriness. In addition, the target quality level and the first quality level can also represent different levels of other quality aspects of the video frame to be identified, such as exposure, color, and clarity. This exemplary embodiment does not specifically limit these aspects. The first aesthetic feature can include two types of aesthetic appeal: a target aesthetic appeal and a first aesthetic appeal. The target aesthetic appeal is higher than the first aesthetic appeal; for example, the target aesthetic appeal represents beauty, while the first aesthetic appeal represents ordinariness.

[0042] In step S120, in response to the first image feature satisfying the preset visual conditions, an image capture operation is triggered.

[0043] In an exemplary embodiment of this disclosure, after obtaining the first screen feature, it can be determined whether the first screen feature meets the preset visual conditions.

[0044] In an optional embodiment, in response to a first screen feature satisfying a preset visual condition, an image capture operation is triggered, including: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetics, an image capture operation is triggered to obtain a captured image, which is then used to trigger an image caching operation.

[0045] When the first quality feature in the first image feature output by the preset first recognition model is the target quality level and the first aesthetic feature is the target aesthetic appeal, it is determined that the first image feature meets the preset visual conditions. Since the resolution of the video frame to be recognized is low, an image capture operation for the video frame to be recognized can be triggered when the first image feature meets the preset visual conditions to obtain the corresponding 4K captured image, and the captured image can also be cached.

[0046] In this exemplary embodiment, the first recognition model can identify the first scene features of the video frame to be recognized, and capture and cache the image when the first scene features indicate that the image is clear and beautiful. This supports aesthetic pre-screening of the video frame to be recognized, and enables real-time perception and capture, achieving the effect of capturing and screening simultaneously. This reduces the generation of invalid data and lowers the waste of storage resources.

[0047] However, whether the captured and cached images are stored or not, it is still possible to continue to identify and confirm the second-view features of the video frame to be identified.

[0048] In an optional embodiment, in response to the first quality feature being a target quality level and the first aesthetic feature including a target aesthetic appeal, the video frame to be identified is input to a preset second recognition model to obtain a second image feature. The second image feature includes a second quality feature and a second aesthetic feature. The second quality feature represents the quality level of at least three types of video frames to be identified, and the second aesthetic feature represents the aesthetic appeal of at least three types of video frames to be identified.

[0049] When the first quality feature in the first image feature output by the preset first recognition model is the target quality level and the first aesthetic feature is the target aesthetics, advanced recognition of the video frame to be recognized can be triggered.

[0050] In an optional embodiment, the second recognition model is deployed on a server.

[0051] When the video frame to be identified triggers aesthetic verification of the second recognition model deployed on the cloud server, the video frame to be identified can be input into the second recognition model so that the second recognition model outputs the second quality feature and the second aesthetic feature.

[0052] In an optional embodiment, the first recognition model is a lightweight recognition model, and the recognition accuracy of the second recognition model is higher than that of the first recognition model.

[0053] Because the first recognition model is a lightweight model, it can output first quality features and first aesthetic features for two types of video frames to be recognized. Correspondingly, the second recognition model has higher recognition accuracy than the first model. Therefore, the second recognition model generally outputs second quality features and second aesthetic features for at least three types of video frames to be recognized. That is, the second recognition model outputs more types of results than the first recognition model, and / or the accuracy of the second recognition model's output is higher than that of the first recognition model.

[0054] The second quality feature may include a target quality level, a first quality level, and a second quality level. The target quality level is higher than the first quality level, and the first quality level is higher than the second quality level. For example, the target quality level represents clarity, the first quality level represents average quality, and the second quality level represents blurriness. In addition, the target quality level, the first quality level, and the second quality level may also represent different levels of other quality aspects of the video frame to be identified, such as exposure, color, and clarity. This exemplary embodiment does not specifically limit this. The second aesthetic feature may include a target aesthetic appeal, a first aesthetic appeal, a second aesthetic appeal, and a third aesthetic appeal. The target aesthetic appeal is higher than the first aesthetic appeal, the first aesthetic appeal is higher than the second aesthetic appeal, and the second aesthetic appeal is higher than the third aesthetic appeal. For example, the target aesthetic appeal represents beauty, the first aesthetic appeal represents flaws, the second aesthetic appeal represents average quality, and the third aesthetic appeal represents sensitivity.

[0055] In an optional embodiment, in response to the second quality feature being the target quality level and the second aesthetic feature being the target aesthetics, an image storage operation of the cached captured image is triggered.

[0056] When the second quality feature in the second image feature output by the preset second recognition model is the target quality level and the second aesthetic feature is the target beauty, it is confirmed that the video frame to be recognized is a clear and beautiful highlight image. Therefore, the captured and cached images corresponding to the video frame to be recognized can be stored.

[0057] In this exemplary embodiment, the second recognition model can store captured and cached images when it determines that the second image features represent a clear and beautiful image. This provides a way to perform secondary screening and aesthetic confirmation, improves the recognition accuracy of the aesthetic effect of the captured images, and can also leverage the dual advantages of the initial screening of the first recognition model on the edge and the precise verification of the second recognition model in the cloud when the second recognition model is deployed on the server. The collaborative work of the edge model and the cloud model also improves the image capture effect.

[0058] In an optional embodiment, an image deletion operation of the cached captured image is triggered in response to a mismatch between the second quality feature and the target quality level, or a mismatch between the second aesthetic feature and the target aesthetic degree.

[0059] When the second quality feature in the second image feature output by the preset second recognition model is not clear, or the second aesthetic feature is not beautiful, it is confirmed that the video frame to be recognized is not a clear and beautiful highlight image. Therefore, the captured and cached images corresponding to the video frame to be recognized can be deleted.

[0060] In this exemplary embodiment, when the image of the second screen feature is unclear or unattractive, the corresponding captured image is deleted, reducing the probability of storing the captured image and further reducing the waste of storage resources, thus improving practicality.

[0061] Furthermore, after storing the captured images representing the highlights, a collection of trip image memories can be generated for the user to display on the vehicle's display device.

[0062] The first and second recognition models used to identify and confirm the video frame to be identified are both trained based on video frame samples.

[0063] In an optional embodiment, video frame samples are acquired, and a first feature sample or a second feature sample of the video frame samples is determined. The first or second feature sample of the video frame sample is verified to obtain the verification result, and the target sample is determined in the video frame sample based on the verification result; A first recognition model is obtained by training the target sample and the first feature sample, or a second recognition model is obtained by training the target sample and the second feature sample.

[0064] The video frame sample can be obtained from a large amount of image data of in-vehicle scenes collected.

[0065] Furthermore, corresponding highlight elements can be preset. For example, in the sky category, highlight elements can be set to blue sky and white clouds, sunrise and sunrise, sunset and sunset, rainbow, lightning, thunderstorm, red moon, Tyndall effect, etc.; in the landform category, highlight elements can be set to mountains, snow-capped mountains, continuous mountain ridges, volcanoes, karst landforms, Danxia landforms, deserts, Gobi deserts, canyons, cliffs, terraces, grasslands, snowfields, etc.; in the water body category, highlight elements can be set to lakes, rivers, wetlands, waterfalls, glaciers, coastlines, beaches, glacial lakes; in the animal category, highlight elements can be set to horses, cattle, sheep, camels, elephants, flocks of birds; in the building category, highlight elements can be set to buildings with a distinctive design, such as CBD (Central Business District). District (Central Business District) buildings, shopping malls, hotels, etc.; religious buildings such as mosques, churches, stupas, giant Buddha statues, halls, etc.; gardens, TV towers; Chinese / classical buildings such as archways, pavilions, ancient city gates, palaces, etc.; European, Russian, Gothic, ancient Greek, ancient Roman, Baroque, Japanese, Mediterranean, and ethnic minority buildings, etc.; in the landscape decoration category, highlight elements can be set as prayer flags, lighting facilities, sculptures, ice sculptures, graffiti walls, etc.

[0066] Furthermore, image retrieval technology and image search technology are used to mine data containing highlight elements in the collected in-vehicle scene image data to obtain video frame samples.

[0067] Furthermore, a multimodal large model can be used to evaluate the image quality and aesthetics of video frame samples according to preset dimensions to obtain the corresponding first feature sample or second feature sample.

[0068] It is worth noting that the first feature sample includes video frame samples that are either clear or blurry in terms of image quality, and that are either average or beautiful in terms of aesthetic evaluation; the second feature sample includes video frame samples that are either clear, average, or blurry in terms of image quality, and that are either average, flawed, sensitive, or beautiful in terms of aesthetic evaluation.

[0069] When evaluating the image quality and aesthetics of video frame samples, corresponding prompts can be constructed according to preset dimensions. These preset dimensions can include technical aspects and image quality, composition and visual guidance, lighting and color, and content and thematic expression.

[0070] Among these, technology and image quality can be further subdivided into sharpness and focus, exposure and brightness, color and white balance, and image clarity. Sharpness and focus require that the subject be sharp, details clearly discernible, and no out-of-focus areas; the image should be free of motion blur; and the lens should be free of dirt and the image should be clear. Exposure and brightness require normal brightness, with no overexposure or loss of detail in highlights, and no noise or black spots in shadows. Color and white balance require that color reproduction be realistic and natural, with no obvious color cast. Image clarity requires the absence of lens flare and blemishes.

[0071] Composition and visual guidance can be further divided into clear subject, simplicity of image, visual balance, and compositional aesthetics. Clear subject requires a well-defined subject with appropriate placement and size. Simplicity of image requires a clean background free of distracting elements. Visual balance requires stable and balanced elements within the image. Compositional aesthetics may involve rhythm, symmetry, the rule of thirds, leading lines, and other compositional trends.

[0072] Light and shadow, along with color, can be further divided into light and shadow quality, tonal control, creative lighting techniques, and color harmony. Light and shadow quality refers to how light shapes the atmosphere of an image and creates a sense of three-dimensionality in objects. Tonal control refers to the natural transitions between highlights, midtones, and shadows. Creative lighting techniques aim to create interesting light and shadow effects and contrasts between light and dark. Color harmony requires natural colors, moderate saturation, and harmonious color temperature.

[0073] Content and theme expression can be further subdivided into theme expression, emotional resonance and storytelling, subject expressiveness, and uniqueness and depth. A clear theme requires the overall visual to convey aesthetics, a story, and emotions. Emotional resonance and storytelling require the ability to evoke emotional responses or associations. Subject expressiveness requires realistic still-life textures and a rich atmosphere. Uniqueness and depth require a unique social, cultural, or personal perspective, expressing profound insights.

[0074] Therefore, multimodal large models can output the quality classification results, aesthetic classification results, and evaluation criteria of video frame samples based on the constructed prompt, which can be used for model training.

[0075] Furthermore, a multimodal large model can be used to verify the video frame samples and their corresponding first or second feature samples, so that the verified samples can be provided for manual review. Once the manual review is completed and confirmed, the final target sample can be obtained.

[0076] At this point, the first recognition model can be trained using the target sample and the first feature sample, and the second recognition model can be trained using SFT (Supervised Fine-Tuning) using the target sample and the second feature sample.

[0077] The image processing method in the embodiments of this disclosure will be described in detail below with reference to an application scenario.

[0078] Figure 2 A flowchart illustrating the image processing method in an application scenario is shown, such as... Figure 2 As shown, in step S210, the input is a video image frame.

[0079] The edge-side perception model can sample the input video image frames at a fixed frame rate.

[0080] In step S220, it is determined whether the cloud migration policy is met.

[0081] The perception model on this edge can be a classification model. When the input is a single video image frame, it can output image quality classification results and image aesthetic classification results. The image quality classification results include two categories: blurry and sharp, and the image aesthetic classification results include two categories: normal and beautiful.

[0082] In step S230, the photo is cached.

[0083] When the image quality classification result of a video image frame is "clear" and the image aesthetic classification result is "beautiful," the capture and caching of the current frame image can be triggered. Therefore, the edge-side perception model can perceive and judge the image quality in real time for aesthetic pre-screening.

[0084] In step S240, it is determined whether there is a highlight.

[0085] When the image quality classification result of a video image frame is "clear" and the image aesthetic classification result is "beautiful," cloud-based recognition can be triggered. This cloud-based model can be a highlight recognition model.

[0086] When video image frames are input into this highlight recognition model, it can output more granular image quality classification results and image aesthetic classification results. The image quality classification results can include three categories: blurry, average, and sharp; the image aesthetic classification results can include four categories: beautiful, flawed, ordinary, and sensitive.

[0087] In step S250, the data is stored.

[0088] When the highlight recognition model confirms that the video image frame is clear and beautiful, the cloud model has completed the aesthetic confirmation step, which can identify the video image frame as a highlight scene and determine to store the corresponding captured image.

[0089] For in-vehicle scenarios, training data can be constructed, cloud-based models can be trained, and edge-based models can be trained by comprehensively considering aspects such as highlight elements, visual quality, and aesthetic evaluation.

[0090] Figure 3The flowchart illustrating the method for training a model in an application scenario is shown, such as... Figure 3 As shown, in step S310, highlight element mining is performed.

[0091] First, an offline dataset can be prepared by collecting a large amount of image data from in-vehicle scenes. Then, image-text retrieval techniques and image search techniques can be used to mine data containing highlight elements from the offline dataset as candidate datasets.

[0092] In step S320, the multimodal large model is evaluated.

[0093] Multimodal large models can construct evaluation prompts for candidate datasets based on preset dimensions, enabling image quality and aesthetic evaluation of the candidate datasets. Therefore, multimodal large models can output quality classification results, aesthetic classification results, and evaluation criteria for candidate datasets, which can be used to train cloud-based and edge-based models.

[0094] In step S330, the large model is validated.

[0095] Furthermore, the multimodal large model can also be used to verify the quality classification results, aesthetic classification results, and evaluation criteria of the candidate dataset to obtain verification results.

[0096] In step S340, manual confirmation is performed.

[0097] If the quality classification results, aesthetic classification results, and evaluation criteria of the candidate dataset pass the verification, it can be provided for manual review. Once confirmed by manual review, the candidate dataset and its corresponding quality and aesthetic classification results can be used as the final training set.

[0098] Figure 4 A flowchart illustrating the method for training a cloud-based model is shown, such as... Figure 4 As shown, the cloud-based model is a multimodal large model, which can be achieved through... Figure 3 The training set shown is obtained by SFT training.

[0099] When training a multimodal large model, if an image is input into the model, it can output quality classification results, aesthetic classification results, and corresponding evaluation criteria. The quality classification results can include three categories: clear, average, and blurry; the aesthetic classification results can include four categories: beautiful, flawed, ordinary, and sensitive; and the corresponding evaluation criteria can include the analysis of subject elements, composition, and color, etc.

[0100] Figure 5 A flowchart illustrating the method for training the edge-side model is shown, as follows: Figure 5 As shown, the edge model is a classification model, which can also be achieved through... Figure 3 The training set shown was obtained through training.

[0101] When training a classification model, if an image is input into the model, the model can output a quality classification result and an aesthetic classification result. The quality classification result can include the probability that the image is blurry and the probability that the image is sharp, while the aesthetic classification result can include the probability that the image is ordinary and the probability that the image is beautiful.

[0102] In the exemplary embodiments of this disclosure, an image capture operation is triggered based on the first frame features of the video frame to be identified, providing an active discrimination and real-time capture method. This achieves the effect of capturing and filtering simultaneously, solving the problem of excessive invalid data in post-screening schemes, reducing excessive time costs and storage resource waste, improving the real-time performance and accuracy of image capture, and eliminating the need for user waiting and operation throughout the entire process, thus optimizing the user experience.

[0103] Furthermore, in an exemplary embodiment of this disclosure, an image processing apparatus is also provided. Figure 6 This is a block diagram of an image processing apparatus according to an exemplary embodiment. (Refer to...) Figure 6 The image processing device 600 includes a feature determination module 610 and an image capturing module 620.

[0104] The feature determination module 610 is configured to acquire a video frame to be identified and determine a first image feature of the video frame to be identified, wherein the first image feature characterizes the image quality and aesthetics of the video frame to be identified. The image capturing module 620 is configured to trigger an image capturing operation in response to the first image feature satisfying a preset visual condition.

[0105] In some embodiments of this disclosure, the feature determination module 610 is configured to: The video frames to be identified are input into a preset first recognition model to obtain a first image feature. The first image feature includes a first quality feature and a first aesthetic feature. The first quality feature represents the quality level of the two types of video frames to be identified, and the first aesthetic feature represents the aesthetics of the two types of video frames to be identified.

[0106] In some embodiments of this disclosure, the image capturing module 620 is configured to: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetics, an image capture operation is triggered to obtain a captured image, which is then used to trigger an image caching operation.

[0107] In some embodiments of this disclosure, the image processing apparatus 600 is further configured to: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetic appeal, the video frame to be identified is input into a preset second recognition model to obtain a second image feature. The second image feature includes a second quality feature and a second aesthetic feature. The second quality feature represents the quality level of at least three types of the video frame to be identified, and the second aesthetic feature represents the aesthetic appeal of at least three types of the video frame to be identified.

[0108] In some embodiments of this disclosure, the second identification model is deployed on a server.

[0109] In some embodiments of this disclosure, the image processing apparatus 600 is further configured to: In response to the second quality feature being the target quality level and the second aesthetic feature being the target aesthetics, an image storage operation is triggered on the cached captured image.

[0110] In some embodiments of this disclosure, the image processing apparatus 600 is further configured to: In response to a mismatch between the second quality feature and the target quality level, or a mismatch between the second aesthetic feature and the target aesthetics, an image deletion operation is triggered on the cached captured image.

[0111] In some embodiments of this disclosure, the first recognition model is a lightweight recognition model, and the recognition accuracy of the second recognition model is higher than that of the first recognition model.

[0112] In some embodiments of this disclosure, the image processing apparatus 600 is further configured to: Acquire video frame samples and determine a first feature sample or a second feature sample of the video frame samples; The first feature sample or the second feature sample of the video frame sample is verified to obtain a verification result, and the target sample is determined in the video frame sample based on the verification result; The first recognition model is obtained by training the target sample and the first feature sample, or the second recognition model is obtained by training the target sample and the second feature sample.

[0113] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0114] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the image processing method provided in this disclosure.

[0115] Figure 7This is a block diagram illustrating a vehicle 700 according to an exemplary embodiment. For example, vehicle 700 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicle. Vehicle 700 may have driver assistance functions.

[0116] Reference Figure 7 The vehicle 700 may include various subsystems, such as an infotainment system 710, a perception system 720, a decision control system 730, a drive system 740, and a computing platform 750. The vehicle 700 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of the vehicle 700 can be interconnected via wired or wireless means.

[0117] In some embodiments, the infotainment system 710 may include a communication system, an entertainment system, and a navigation system, etc.

[0118] The perception system 720 may include several sensors for sensing information about the environment surrounding the vehicle 700. For example, the perception system 720 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0119] The decision control system 730 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0120] The drive system 740 may include components that provide powered motion to the vehicle 700. In one embodiment, the drive system 740 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0121] Some or all of the functions of vehicle 700 are controlled by computing platform 750. Computing platform 750 may include at least one processor 751 and memory 752, and processor 751 may execute instructions 753 stored in memory 752.

[0122] The processor 751 can be any conventional processor, such as a commercially available CPU. The processor may also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems on chips (SOCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0123] The memory 752 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0124] In addition to instruction 753, memory 752 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 752 can be used by computing platform 750.

[0125] In this embodiment of the disclosure, processor 751 may execute instruction 753 to complete all or part of the steps of the above-described image processing method.

[0126] Figure 8 This is a block diagram illustrating an image processing apparatus 800 according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0127] Reference Figure 8 The device 800 may include one or more of the following components: a first processing component 802, a first memory 804, a first power supply component 806, a multimedia component 808, an audio component 810, a first input / output interface 812, a sensor component 814, and a communication component 816.

[0128] The first processing component 802 typically controls the overall operation of the device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. The first processing component 802 may include one or more first processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, the first processing component 802 may include one or more modules to facilitate interaction between the first processing component 802 and other components. For example, the first processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the first processing component 802.

[0129] The first memory 804 is configured to store various types of data to support the operation of the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phonebook data, messages, pictures, videos, etc. The first memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0130] The first power supply component 806 provides power to various components of the device 800. The first power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 800.

[0131] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0132] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in first memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0133] The first input / output interface 812 provides an interface between the first processing component 802 and the peripheral interface module, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0134] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0135] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0136] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0137] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a first memory 804 including instructions that can be executed by a first processor 820 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0138] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the image processing method described above when executed by the programmable device.

[0139] Figure 9 This is a block diagram illustrating an image processing apparatus 900 according to an exemplary embodiment. For example, apparatus 900 may be provided as a server. (Refer to...) Figure 9 The apparatus 900 includes a second processing component 922, which further includes one or more processors, and memory resources represented by a second memory 932 for storing instructions, such as application programs, that can be executed by the second processing component 922. The application programs stored in the second memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the second processing component 922 is configured to execute instructions to perform the methods described above.

[0140] The device 900 may further include a second power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and a second input / output interface 958. The device 900 can operate on an operating system stored in a second memory 932.

[0141] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0142] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0143] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0144] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: The video frame to be identified is acquired, and a first image feature of the video frame to be identified is determined. The first image feature characterizes the image quality and aesthetics of the video frame to be identified. In response to the first image feature satisfying the preset visual conditions, an image capture operation is triggered.

2. The image processing method according to claim 1, characterized in that, The determination of the first frame feature of the video frame to be identified includes: The video frames to be identified are input into a preset first recognition model to obtain a first image feature. The first image feature includes a first quality feature and a first aesthetic feature. The first quality feature represents the quality level of the two types of video frames to be identified, and the first aesthetic feature represents the aesthetics of the two types of video frames to be identified.

3. The image processing method according to claim 2, characterized in that, The step of triggering an image capture operation in response to the first image feature satisfying a preset visual condition includes: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetics, an image capture operation is triggered to obtain a captured image, which is then used to trigger an image caching operation.

4. The image processing method according to claim 3, characterized in that, The method further includes: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetic appeal, the video frame to be identified is input into a preset second recognition model to obtain a second image feature. The second image feature includes a second quality feature and a second aesthetic feature. The second quality feature represents the quality level of at least three types of the video frame to be identified, and the second aesthetic feature represents the aesthetic appeal of at least three types of the video frame to be identified.

5. The image processing method according to claim 4, characterized in that, The second recognition model is deployed on a server.

6. The image processing method according to claim 4, characterized in that, The method further includes: In response to the second quality feature being the target quality level and the second aesthetic feature being the target aesthetics, an image storage operation is triggered on the cached captured image.

7. The image processing method according to claim 4, characterized in that, The method further includes: In response to a mismatch between the second quality feature and the target quality level, or a mismatch between the second aesthetic feature and the target aesthetics, an image deletion operation is triggered on the cached captured image.

8. The image processing method according to claim 4, characterized in that, The first recognition model is a lightweight recognition model, and the recognition accuracy of the second recognition model is higher than that of the first recognition model.

9. The image processing method according to claim 2 or 4, characterized in that, The method further includes: Acquire video frame samples and determine a first feature sample or a second feature sample of the video frame samples; The first feature sample or the second feature sample of the video frame sample is verified to obtain a verification result, and the target sample is determined in the video frame sample based on the verification result; The first recognition model is obtained by training the target sample and the first feature sample, or the second recognition model is obtained by training the target sample and the second feature sample.

10. An image processing apparatus, characterized in that, include: The feature determination module is configured to acquire a video frame to be identified and determine a first image feature of the video frame to be identified, wherein the first image feature characterizes the image quality and aesthetics of the video frame to be identified. The image capture module is configured to trigger an image capture operation in response to the first image feature meeting a preset visual condition.

11. The image processing apparatus according to claim 10, characterized in that, The feature determination module is configured as follows: The video frames to be identified are input into a preset first recognition model to obtain a first image feature. The first image feature includes a first quality feature and a first aesthetic feature. The first quality feature represents the quality level of the two types of video frames to be identified, and the first aesthetic feature represents the aesthetics of the two types of video frames to be identified.

12. The image processing apparatus according to claim 11, characterized in that, The image capturing module is configured as follows: In response to the first quality feature being the target quality level and the first aesthetic feature being the target aesthetics, an image capture operation is triggered to obtain a captured image, which is then used to trigger an image caching operation.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 9.

14. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.

15. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute executable instructions stored in the memory to implement the steps of the image processing method according to any one of claims 1 to 9.

Images