Vehicle double-flash image recognition method and system based on data enhancement

Abstract

The invention provides a vehicle double-flash image recognition method and system based on data enhancement. The method comprises the following steps: acquiring video data obtained by shooting a vehicle; performing frame extraction on the video data to obtain an image set; inputting the image set into a preset identification model to identify whether the vehicle turns on double flash, and obtaining an identification result; wherein the recognition model is a deep learning model; training samples used in the process of training the identification model are obtained by adopting a data enhancement technology. Therefore, the double-flash of the vehicle can be quickly identified.

Description

Technical Field

[0001] The present invention relates to the technical field of online car-hailing operation management, and in particular to a vehicle double-flash image recognition method and system based on data enhancement. Background Art

[0002] In the online car-hailing industry, vehicle safety status monitoring is a crucial link, especially the status of the vehicle's double flash lights, which is directly related to road safety and emergency response. Double flash lights, also known as emergency lights or hazard warning lights, are signal lights used when a vehicle encounters an emergency or needs to stop temporarily to alert other road users. However, existing technologies have some limitations in real-time monitoring and identification of the status of vehicle double flash lights.

[0003] There is currently a lack of a quick and convenient way to monitor double flashes. Summary of the invention

[0004] To this end, the present invention provides a vehicle double flash image recognition method and system based on data enhancement, aiming to solve the technical problems in the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions:

[0006] According to a first aspect of the present invention, the present invention provides a vehicle double flash image recognition method based on data enhancement, the method comprising:

[0007] Acquire video data captured by the vehicle;

[0008] Extracting frames from the video data to obtain an image set;

[0009] Inputting the image set into a preset recognition model to recognize whether the vehicle has turned on the double flash, and obtaining a recognition result;

[0010] Among them, the recognition model is a deep learning model; the training samples used in the process of training the recognition model are obtained by adopting data enhancement technology.

[0011] In some embodiments, extracting frames from the video data to obtain an image set includes:

[0012] Extract frames from the video every N frames to obtain an image set;

[0013] Wherein, N is a positive integer.

[0014] In some embodiments, the step of extracting frames from the video data to obtain an image set further comprises:

[0015] The collected image set is standardized so that the pictures in the image set meet the preset requirements.

[0016] In some embodiments, the training process of the recognition model includes:

[0017] Get the original image data;

[0018] Performing standardization processing on the original image data;

[0019] Annotating the original image data to obtain a first part of training samples;

[0020] Perform image enhancement processing on part of the original image data and add annotations to obtain the second part of training samples;

[0021] Summarize the training samples in the first part and the training samples in the second part to obtain training samples;

[0022] The recognition model is trained based on the training samples.

[0023] In some embodiments, the vehicle double flash image recognition method based on data enhancement is applied to a mobile phone.

[0024] In some embodiments, the obtaining of video data captured by the vehicle includes:

[0025] The vehicle is photographed by a camera device on the mobile phone to obtain video data.

[0026] According to a second aspect of the present invention, the present invention provides a vehicle double flash image recognition system based on data enhancement, characterized in that it includes: a mobile phone terminal and a remote cloud;

[0027] The mobile phone is used to shoot the vehicle based on the camera device of the mobile phone, obtain video data, and send it to the remote cloud;

[0028] The remote cloud is used to obtain video data taken of the vehicle; extract frames from the video data to obtain an image set; input the image set into a preset recognition model to identify whether the vehicle has turned on the double flash, and obtain a recognition result; wherein the recognition model is a deep learning model; the training samples used in the process of training the recognition model are obtained using data enhancement technology.

[0029] According to a third aspect of the present invention, the present invention provides a vehicle double flash image recognition device based on data enhancement, characterized in that it includes:

[0030] An acquisition module is used to acquire video data captured by the vehicle;

[0031] A frame extraction module, used for extracting frames from the video data to obtain an image set;

[0032] A recognition module, used for inputting the image set into a preset recognition model to recognize whether the vehicle has turned on the double flash, and obtaining a recognition result;

[0033] Among them, the recognition model is a deep learning model; the training samples used in the process of training the recognition model are obtained by adopting data enhancement technology.

[0034] According to a fourth aspect of the present invention, the present invention provides an electronic device, characterized in that it includes:

[0035] A processor, and a memory for storing a program executable by the processor;

[0036] The processor is used to implement the above-mentioned vehicle double flash image recognition method based on data enhancement by running the program in the memory.

[0037] According to the fifth aspect of the present invention, the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the processor executes the vehicle double-flash image recognition method based on data enhancement as described above.

[0038] The present invention adopts the above technical solution and has at least the following beneficial effects:

[0039] Through the scheme of the present invention, the accuracy of identifying the status of the double flash lights can be greatly improved by using a deep learning model for image recognition. The deep learning model can learn complex feature representations from a large amount of data, thereby more accurately distinguishing the on and off states of the double flash lights. Through data enhancement technology, more training samples can be generated, which helps the model learn more generalized features and improves the model's recognition ability under different conditions, especially in terms of lighting changes, weather conditions, and vehicle type diversity. By extracting frames from video data, the amount of data processed can be reduced while retaining key information and improving processing efficiency.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1A schematic diagram of the process of a vehicle double flash image recognition method based on data enhancement provided by an embodiment of the present invention is shown;

[0043] Figure 2 A schematic structural diagram of a vehicle double flash image recognition system based on data enhancement provided by another embodiment of the present invention is shown;

[0044] Figure 3 It is a structural schematic diagram of a vehicle double-flash image recognition device based on data enhancement provided in one embodiment of the present application.

[0045] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0047] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0048] The embodiment of the present invention provides a vehicle double flash image recognition method based on data enhancement, such as Figure 1 As shown, at least the following steps S101 to S103 may be included:

[0049] Step S101, obtaining video data captured by the vehicle;

[0050] This step is the starting point of the entire double flash light recognition process, which involves using a camera device (such as a vehicle's built-in camera or a mobile phone camera) to shoot the vehicle in real time to obtain video data. The video data contains the real-time status information of the vehicle, including the status of the vehicle's double flash lights. The key to this step is to ensure the quality and continuity of the video data so that the subsequent steps can effectively extract useful information from it.

[0051] The detailed operations include: selecting a suitable camera device and ensuring that it can cover the position of the vehicle's double flash lights. Adjusting the camera device parameters, such as resolution and frame rate, to ensure the clarity and smoothness of the video data. Starting the camera device to shoot the vehicle in real time and storing the captured video data for subsequent processing.

[0052] Step S102, extracting frames from the video data to obtain an image set;

[0053] In this step, key frames are extracted from the continuous video stream to form an image set. These images will be used for subsequent double flash status recognition. The purpose of frame extraction is to reduce the amount of data while retaining enough information for accurate recognition.

[0054] Detailed operation: Determine the frame extraction strategy, for example, extract one frame every N frames, where N is a preset positive integer. Analyze the video data frame by frame and select key frames according to the frame extraction strategy. Store the extracted frames as an image set, which will be used in the subsequent recognition process.

[0055] Step S103, inputting the image set into a preset recognition model to recognize whether the vehicle has turned on the hazard lights, and obtaining a recognition result;

[0056] Among them, the recognition model is a deep learning model; the training samples used in the process of training the recognition model are obtained by adopting data enhancement technology.

[0057] This step is the core of the entire process and involves using a deep learning model to analyze each image in the image set to identify whether the vehicle’s hazard lights are turned on.

[0058] Detailed operation: Prepare a pre-trained deep learning model that can recognize the features of the double flash lights. Input each image in the image set into the deep learning model. The model analyzes each image, extracts the features of the double flash lights, and classifies them. The model outputs the recognition results of each image, including the probability or confidence of whether the vehicle has turned on the double flash lights.

[0059] Application of deep learning models and data enhancement technology: Deep learning model: Select a deep learning architecture suitable for image recognition tasks, such as a convolutional neural network (CNN). The model can automatically learn features from images to distinguish between the on and off states of the dual flash lights. Data enhancement technology: In the process of training the deep learning model, data enhancement technology is used to expand the training samples. This includes rotating, scaling, and color adjustment of the original image to simulate different shooting conditions and environmental changes. Data enhancement helps to improve the generalization ability of the model, enabling it to more accurately identify the dual flash light status in practical applications. Through these three steps, efficient and accurate recognition of the dual flash light status of the vehicle can be achieved, providing technical support for road safety and vehicle management.

[0060] In some embodiments, extracting frames from the video data to obtain an image set includes:

[0061] The video is extracted every N frames to obtain an image set, where N is a positive integer.

[0062] In the present invention, the purpose of frame extraction is to extract an image set containing the double flash light status information from the video, which is then used for recognition by the deep learning model. The following is a detailed description of the frame extraction process:

[0063] N is a positive integer, representing the interval at which a frame is extracted from every N frames of the video as a key frame. The choice of N depends on the frame rate of the video and the frequency of the events that need to be identified. If the status of the flash light changes frequently, a smaller N value can be selected to capture more details; if the changes are less frequent, a larger N value can be selected to reduce the amount of data processed. Of course, it also depends on the frame rate of the video captured by the shooting device.

[0064] Take the captured video data as input, which can be real-time or recorded video files. Read the frame rate information of the video to determine how many frames of images there are per second;

[0065] Set a counter to track the number of the currently processed video frame. Traverse the video frames: Traverse the video data frame by frame from the beginning to the end of the video. Frame number judgment: For each frame, check whether its number is a multiple of N. If so, extract the frame as a key frame; if not, skip the frame. Store all extracted key frames as an image set, which will be used for subsequent double flash state recognition.

[0066] In some embodiments, the step of extracting frames from the video data to obtain an image set further includes: standardizing the collected image set so that the images in the image set meet preset requirements.

[0067] For example, the collected image data is scaled according to the ratio of the long side, so that the longest side reaches 512 pixels, while keeping the aspect ratio of the image unchanged. Next, the image is normalized and the pixel data is adjusted to have a mean of 0 and a variance of 1. Finally, the image is padded to fill the image size to a multiple of 32 to ensure that it meets the model step size requirements.

[0068] The training process of the recognition model includes:

[0069] Acquire original image data; perform standardization processing on the original image data; annotate the original image data to obtain training samples of the first part; perform image enhancement processing on part of the original image data, and add annotations to obtain training samples of the second part; summarize the training samples of the first part and the training samples of the second part to obtain training samples; train the recognition model based on the training samples.

[0070] Among them, the original image data is annotated to obtain the first part of the training samples, including: Annotating the double flash light status: Manually annotating each image to determine whether the double flash lights of the vehicle in the image are turned on or off. Creating labels: Creating corresponding labels for each image, which will be used to supervise the learning process of the model. Image enhancement processing is performed on part of the original image data, and annotations are added to obtain the second part of the training samples, including: Image enhancement: Applying data enhancement techniques to some of the original images, such as rotation, scaling, color adjustment, etc., to generate new image samples. Enhanced annotation: Performing the same annotation processing on the enhanced images as the original images to ensure that these images also have correct labels.

[0071] The recognition model is trained based on the training samples, including: Model selection: selecting a suitable deep learning model architecture, such as a convolutional neural network (CNN). Training configuration: setting training parameters, including learning rate, batch size, number of iterations, etc. Model training: using the training sample set to train the model, and optimizing the model weights through the back propagation algorithm. Performance evaluation: evaluating the performance of the model on the validation set, such as accuracy, precision, recall and other indicators. Model tuning: adjusting the model structure or training parameters based on the performance evaluation results to improve the generalization ability of the model. Through the above steps, a high-quality training sample set can be constructed, and a deep learning model that can accurately identify the status of the vehicle's double flash lights can be trained. This process involves not only data preparation and preprocessing, but also model training and optimization to ensure the effectiveness and robustness of the model in practical applications.

[0072] In practical applications, the vehicle double flash image recognition method based on data enhancement is applied to a mobile phone. Correspondingly, the acquisition of video data obtained by shooting the vehicle includes: shooting the vehicle based on a camera device on the mobile phone to obtain video data.

[0073] Reference Figure 2 , the solution provided by this application is described below in conjunction with a specific embodiment: In view of the problem that the existing solution is difficult to identify the vehicle double flash image when deployed on the edge, we propose a vehicle double flash image method based on data enhancement for the online car-hailing registration scenario. The detailed technical solution is as follows:

[0074] (1) Video frame extraction module

[0075] Extract vehicle inspection video data submitted by drivers from the user database. All data includes double flash. Extract frames every 10 frames of the video. Finally, only when two images of lights on and lights off are detected can the double flash be considered to be turned on.

[0076] (2) Image standardization processing module

[0077] First, the collected image data is scaled according to the ratio of the long side, so that the longest side reaches 512 pixels, while keeping the aspect ratio of the image unchanged. Then, the image is normalized and the pixel data is adjusted to have a mean of 0 and a variance of 1. Finally, the image is padded to fill the image size to a multiple of 32 to ensure that it meets the model step size requirements.

[0078] (3) Image detection module

[0079] The image detection module is trained through data annotation and yolov5n. This patent uses the lightest yolov5n algorithm in the existing yolov5 series, and the model weight is only 3Mb.

[0080] (4) Enhanced flash data module

[0081] In order to give the pictures of non-flashing headlights a vivid flashing effect, we designed a glow effect generation program based on Python. The program can accurately locate the position of the headlights in the picture and create a series of GIF image sequences with dynamically changing light colors. Through these images, we can randomly generate yellow halos and lighting effects, perfectly simulating the real scene of double flashing headlights. After labeling, these enhanced data are used to train yolov5n.

[0082] (5) Edge Quantization Compression Module

[0083] We use quantization tools to quantize the feature layer into onnx format and deploy it on the mobile phone. However, the traditional pytorch code can only convert the previous classification layer into an acceleration format that can be called by cuda (mobile GPU). The Softmax function in the last layer calls the mobile phone CPU, which causes the edge prediction time to rise from 4ms to 40ms for a single image. Therefore, we optimize it here and encapsulate the Softmax function into a classification layer written by CUDA, so that the final prediction time returns to 4ms.

[0084] (6) Audit result judgment module

[0085] After normalizing the image and passing it through double flash recognition, we can obtain the confidence that the image is a double flash image. By setting different confidence thresholds on the test set, we can generate precision-recall (PR) curves under different thresholds. According to actual business needs, we can choose a suitable threshold to achieve the best balance between precision and recall. Ultimately, this threshold will help us accurately determine whether the image is a double flash.

[0086] Further, as Figure 1 In a specific implementation, an embodiment of the present invention provides a vehicle double flash image recognition system based on data enhancement, including: a mobile phone terminal and a remote cloud;

[0087] The mobile phone is used to shoot the vehicle based on the camera device of the mobile phone, obtain video data, and send it to the remote cloud;

[0088] The remote cloud is used to execute the above-mentioned vehicle double-flash image recognition method based on data enhancement. The specific vehicle double-flash image recognition method based on data enhancement includes: acquiring video data captured by the vehicle; extracting frames from the video data to obtain an image set; inputting the image set into a preset recognition model to identify whether the vehicle has turned on the double flash to obtain a recognition result; wherein the recognition model is a deep learning model; and the training samples used in the process of training the recognition model are obtained using data enhancement technology.

[0089] Further, as Figure 1 The specific implementation of the present invention provides a vehicle double flash image recognition device based on data enhancement, such as Figure 3 The vehicle double flash image recognition device based on data enhancement includes:

[0090] An acquisition module 31 is used to acquire video data captured by the vehicle;

[0091] A frame extraction module 32, used for extracting frames from the video data to obtain an image set;

[0092] The recognition module 33 is used to input the image set into a preset recognition model to recognize whether the vehicle has turned on the double flash, and obtain a recognition result;

[0093] Among them, the recognition model is a deep learning model; the training samples used in the process of training the recognition model are obtained by adopting data enhancement technology.

[0094] Below, reference Figure 4 To describe an electronic device according to an embodiment of the present application. Figure 4 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0095] like Figure 4 As shown, electronic device 400 includes one or more processors 410 and memory 420 .

[0096] The processor 410 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0097] The memory 420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 410 may run the program instructions to implement the data security analysis method of each embodiment of the present application described above and / or other desired functions. Various contents such as category correspondences may also be stored in the computer-readable storage medium.

[0098] In one example, the electronic device 400 may further include: an input device 430 and an output device 440 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0099] In addition, the input device 430 may also include, for example, a keyboard, a mouse, an interface, etc. The output device 440 may output various information to the outside, including analysis results, etc. The output device 440 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0100] Of course, to simplify, Figure 4Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application conditions.

[0101] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the data security analysis method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0102] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0103] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the data security analysis method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0104] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0105] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A vehicle double flash image recognition method based on data enhancement, characterized in that: The method comprises: Acquire video data captured by the vehicle; Extracting frames from the video data to obtain an image set; Inputting the image set into a preset recognition model to recognize whether the vehicle has turned on the double flash, and obtaining a recognition result; Among them, the recognition model is a deep learning model; the training samples used in the process of training the recognition model are obtained by adopting data enhancement technology.

2. The method according to claim 1, characterized in that The step of extracting frames from the video data to obtain an image set includes: Extract frames from the video every N frames to obtain an image set; Wherein, N is a positive integer.

3. The method according to claim 1, characterized in that The video data is extracted to obtain an image set, and then the following steps are further included: The collected image set is standardized so that the pictures in the image set meet the preset requirements.

4. The method according to claim 1, characterized in that: The training process of the recognition model includes: Get the original image data; Performing standardization processing on the original image data; Annotating the original image data to obtain a first part of training samples; Perform image enhancement processing on part of the original image data and add annotations to obtain the second part of training samples; Summarize the training samples in the first part and the training samples in the second part to obtain training samples; The recognition model is trained based on the training samples.

5. The method according to claim 1, characterized in that The vehicle double flash image recognition method based on data enhancement is applied to a mobile phone.

6. The method according to claim 5, characterized in that The step of obtaining video data captured by the vehicle includes: The vehicle is photographed by a camera device on the mobile phone to obtain video data.

7. A vehicle double flash image recognition system based on data enhancement, characterized in that: include: Mobile phone and remote cloud; The mobile phone is used to shoot the vehicle based on the camera device of the mobile phone, obtain video data, and send it to the remote cloud; The remote cloud is used to obtain video data captured by the vehicle; Extract frames from the video data to obtain an image set; input the image set into a preset recognition model to recognize whether the vehicle has turned on the double flash, and obtain a recognition result; wherein the recognition model is a deep learning model; The training samples used in the process of training the recognition model are obtained by adopting data enhancement technology.

8. A vehicle double flash image recognition device based on data enhancement, characterized in that: include: An acquisition module is used to acquire video data captured by the vehicle; A frame extraction module, used for extracting frames from the video data to obtain an image set; A recognition module, used for inputting the image set into a preset recognition model to recognize whether the vehicle has turned on the double flash, and obtaining a recognition result; Wherein, the recognition model is a deep learning model; The training samples used in the process of training the recognition model are obtained by adopting data enhancement technology.

9. An electronic device, characterized in that: include: A processor, and a memory for storing a program executable by the processor; The processor is used to implement the vehicle double flash image recognition method based on data enhancement as described in any one of claims 1 to 6 by running the program in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to execute the vehicle double-flash image recognition method based on data enhancement as claimed in any one of claims 1 to 6.

Images