Image recognition method, device and system and terminal equipment

Abstract

The invention relates to an image recognition method, device and system and terminal equipment, the method is applied to a second terminal in the image recognition system, the image recognition system comprises a first terminal, an image injection module, a controller and the second terminal, and the second terminal is connected with the first terminal, the image injection module and the controller. The first terminal is connected with the image injection module, the image injection module is connected with the controller, and the method comprises the following steps: acquiring first image data recognized by the first terminal for a to-be-recognized image and second image data recognized by the controller for the to-be-recognized image; under the condition that the second image data and the first image data do not meet the preset condition through comparison, parameters of the image injection module are adjusted, so that the first image data and the second image data meet the preset condition. Therefore, the parameters of the image injection module in the image recognition system can be calibrated online, the image recognition accuracy of the controller is improved, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular, to an image recognition method, device, system, and terminal device. Background Art

[0002] Currently, in the camera image simulation in the field of autonomous driving simulation testing, generally, an image injection device is used to simulate a camera to input an image to a controller, so that the controller can recognize the input image.

[0003] In practical applications, in the traditional form of camera image injection, after the parameter of the injection device is debugged by the provider of the injection device, it is directly delivered to the engineer for simulation testing. This will result in that when the camera parameter changes or the image recognition program inside the intelligent driving controller is upgraded, the current image injection device cannot meet the changed usage requirements, resulting in a large difference between the image recognized by the controller and the image input by the image injection device, affecting the user experience. Summary of the Invention

[0004] This application provides an image recognition method, device, system, and terminal device to solve the technical problem in the prior art that when the camera parameter changes or the image recognition program inside the intelligent driving controller is upgraded, the current image injection device cannot meet the changed usage requirements, resulting in a large difference between the image recognized by the controller and the image input by the image injection device, affecting the user experience.

[0005] In a first aspect, this application provides an image recognition method, which is applied to a second terminal in an image recognition system. The image recognition system includes a first terminal, an image injection module, a controller, and a second terminal. The second terminal is respectively connected to the first terminal, the image injection module, and the controller. The first terminal is connected to the image injection module, and the image injection module is connected to the controller. The method includes:

[0006] Obtain first image data obtained by the first terminal for recognizing an image to be recognized, and second image data obtained by the controller for recognizing the image to be recognized;

[0007] Wherein, the first terminal sends the first image data to the image injection module for processing to obtain third image data, and the image injection module sends the third image data to the controller for processing to obtain the second image data;

[0008] In the case where it is determined that the second image data and the first image data do not meet a preset condition, adjust the parameters of the image injection module so that the first image data and the second image data meet the preset condition.

[0009] As an optional implementation, after adjusting the parameters of the image injection module, the following steps are further included:

[0010] Restart the image injection module;

[0011] Re-obtain the first image data of the first terminal for recognizing the image to be recognized, and the second image data of the controller for recognizing the image to be recognized;

[0012] In the case where it is determined that the first image data and the second image data do not meet the preset conditions, return to execute the step of adjusting the parameters of the image injection module until the first image data and the second image data meet the preset conditions.

[0013] As an optional implementation, obtaining the first image data of the first terminal for recognizing the image to be recognized, and the second image data of the controller for recognizing the image to be recognized includes:

[0014] Obtain the first image data of the image to be recognized saved by the first terminal through a preset first remote control instruction;

[0015] Obtain the processed image of the controller through a preset second remote control instruction;

[0016] Recognize the processed image to obtain the second image data.

[0017] As an optional implementation, the image to be recognized is a preset color card image, the first image data is the first pixel value of the color card image, and the second image data is the second pixel value of the color card image;

[0018] The situation where the second image data and the first image data do not meet the preset conditions includes:

[0019] Compare whether the first pixel value and the second pixel value are consistent;

[0020] In the case where it is determined that the first pixel value and the second pixel value are inconsistent, determine that the first image data and the second image data do not meet the preset conditions.

[0021] As an optional implementation, adjusting the parameters of the image injection module includes:

[0022] Configure the network parameters of the image injection module through a remote control instruction;

[0023] In the case where it is determined that normal communication with the image injection module is established, obtain the image parameter configuration of the image injection module;

[0024] Configure the image parameters in the image injection module according to the configured image parameters.

[0025] When it is determined that the color adjustment function is currently enabled, obtain the corresponding adjustment matrix and offset matrix, and input the adjustment matrix and the offset matrix into the image injection module.

[0026] As an optional implementation manner, the image injection module processes the received first image data in the following manner to obtain third image data:

[0027] Obtain the current adjustment matrix and offset matrix;

[0028] Process the first image data according to the adjustment matrix and the offset matrix to obtain the third image data.

[0029] As an optional implementation manner, the first terminal includes a first graphics processor, the first graphics processor is connected to the image injection module through a first preset interface, the first graphics processor is connected to a preset first display module through a second preset interface, and the first display module is used to display the first image data;

[0030] The second terminal includes a second graphics processor, the second graphics processor is connected to a preset second display module through a third preset interface, and the second display module is used to display the first image data and the second image data.

[0031] In a second aspect, the present application provides an image recognition device, which is applied to a second terminal in an image recognition system. The image recognition system includes a first terminal, an image injection module, a controller, and a second terminal. The second terminal is respectively connected to the first terminal, the image injection module, and the controller. The first terminal is connected to the image injection module, and the image injection module is connected to the controller. The device includes:

[0032] An acquisition module, configured to acquire first image data obtained by the first terminal for recognizing an image to be recognized, and second image data obtained by the controller for recognizing the image to be recognized;

[0033] Wherein, the first terminal sends the first image data to the image injection module for processing to obtain third image data, and the image injection module sends the third image data to the controller for processing to obtain the second image data;

[0034] An adjustment module, configured to adjust parameters of the image injection module when it is determined that the second image data and the first image data do not meet a preset condition, so that the first image data and the second image data meet the preset condition.

[0035] In a third aspect, the present application provides an image recognition system, including: a first terminal, an image injection module, a controller, and a second terminal; the second terminal is respectively connected to the first terminal, the image injection module, and the controller, the first terminal is connected to the image injection module, and the image injection module is connected to the controller;

[0036] The first terminal is configured to store an image to be recognized, recognize the image to be recognized to obtain first image data, and send the first image data to the image injection module;

[0037] The image injection module is configured to receive the first image data, process the first image data to obtain third image data, and send the third image data to the controller;

[0038] The controller is configured to receive the third image data and process the third image data to obtain second image data;

[0039] The second terminal is configured to execute any one of the image recognition methods in the first aspect.

[0040] In a fourth aspect, the present application provides a terminal device, including: a processor and a memory, and the processor is configured to execute an image recognition program stored in the memory to implement any one of the image recognition methods in the first aspect.

[0041] In a fifth aspect, the present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any one of the image recognition methods in the first aspect.

[0042] The technical solution provided by the embodiments of the present application obtains the first image data obtained by the first terminal for recognizing the image to be recognized, and the second image data obtained by the controller for recognizing the above-mentioned image to be recognized; wherein, the first terminal sends the first image data to the image injection module for processing to obtain the third image data, and the image injection module can send the third image data to the controller for processing to obtain the second image data. In the case where it is determined that the second image data and the first image data do not meet the preset conditions, the parameters of the image injection module are adjusted so that the first image data and the second image data meet the preset conditions. In this technical solution, during the process of the image recognition system recognizing an image, the second control terminal respectively obtains the image data obtained by the first terminal and the controller for recognizing the image to be recognized, and in the case where it is determined that the image data recognized by the first terminal and the controller for the image to be recognized is inconsistent, the parameters of the image injection module can be adjusted online to make the image data recognized by the controller consistent with the image data recognized by the first terminal, ensuring the accuracy of the controller's recognition, realizing the online calibration of the parameters of the image injection module in the image recognition system, improving the accuracy of the controller's image recognition, and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application and, together with the specification, are used to explain the principles of the present application.

[0044] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments or the prior art text. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] One or more embodiments are illustrated by way of example in the accompanying drawings corresponding thereto. These exemplary illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, unless otherwise stated, and the drawings in the figures do not constitute a scale limitation.

[0046] Figure 1 It is a schematic structural diagram of an image recognition system provided by an embodiment of the present application;

[0047] Figure 2 It is a schematic structural diagram of another image recognition system provided by an embodiment of the present application;

[0048] Figure 3 It is a flowchart of an embodiment of an image recognition method provided by an embodiment of the present application;

[0049] Figure 4A flowchart of an embodiment of another image recognition method provided by an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a color card image provided by an embodiment of the present application;

[0051] Figure 6 A flowchart of an embodiment of yet another image recognition method provided by an embodiment of the present application;

[0052] Figure 7 A flowchart of an embodiment of still another image recognition method provided by an embodiment of the present application;

[0053] Figure 8 A block diagram of an embodiment of an image recognition device provided by an embodiment of the present application;

[0054] Figure 9 A schematic diagram of the structure of a terminal device provided by an embodiment of the present application. Detailed implementation manners

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part rather than all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0056] The following disclosure provides many different embodiments or examples for implementing different structures of the present application. To simplify the disclosure of the present application, components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present application. In addition, the present application may repeat reference numerals and / or letters in different examples. Such repetition is for the purpose of simplification and clarity and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0057] To solve the technical problem in the prior art that when the camera parameters change or the image recognition program inside the intelligent driving controller is upgraded, the current image injection device cannot meet the changed usage requirements, resulting in a large difference between the image recognized by the controller and the image input by the image injection device, which affects the user experience, this application provides an image recognition method, device, system, and terminal device. During the process of image recognition by the image recognition system, the second control terminal can respectively obtain the image data of the first terminal and the controller for recognizing the image to be recognized. When it is determined that the image data recognized by the first terminal and the controller for the image to be recognized is inconsistent, the parameters of the image injection module can be adjusted online to make the image data recognized by the controller consistent with the image data recognized by the first terminal, ensuring the accuracy of the controller's recognition, realizing the online calibration of the parameters of the image injection module in the image recognition system, improving the accuracy of the controller's image recognition, and enhancing the user experience.

[0058] To facilitate the understanding of the image recognition method provided in the embodiments of this application, the following first gives an example of the image recognition system involved in the image recognition method provided in this application.

[0059] See Figure 1 , which is a schematic structural diagram of an image recognition system provided in the embodiments of this application. As Figure 1 shown, the image recognition system 10 may include a first terminal 11, an image injection module 12, a controller 13, and a second terminal 14.

[0060] Among them, the above-mentioned first terminal 11 can be a hardware device or software that supports network connection to provide various network services. When the first terminal 11 is hardware, it can be various electronic devices supporting a display screen, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the first terminal 11 is software, it can be installed in the above-listed electronic devices.

[0061] The above-mentioned image injection module 12 is used to inject the received image to be recognized into the internal for recognition and send the recognized image data to the controller 13. The image injection module 12 can be a video injection chassis or other image injection modules, and the embodiments of this application do not limit this.

[0062] The above-mentioned controller 13 refers to an ECU (Electronic Control Unit), which can be an intelligent driving controller in a vehicle or other controllers for image recognition, and the embodiments of this application do not limit this.

[0063] The second terminal 14 may be a hardware device or software that supports network connection and provides various network services. When the second terminal 14 is hardware, it may support various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the second terminal 14 is software, it may be installed in the electronic devices listed above.

[0064] In some embodiments of the present application, the first terminal 11 may be connected to the image injection module 12, the image injection module 12 may be connected to the controller 13, and the second terminal 14 may be connected to the first terminal 11, the image injection module 12, the controller 13, and the second terminal 14 respectively.

[0065] The first terminal 11 may be used to store the image to be identified, identify the image to be identified, obtain first image data, and send the first image data to the image injection module 12 .

[0066] The image injection module 12 may be configured to receive first image data, process the first image data to obtain third image data, and send the third image data to the controller 13 .

[0067] The controller 13 may be configured to receive the third image data and process the third image data to obtain the second image data.

[0068] The above-mentioned second terminal can be used as the execution subject of the embodiment of the present application, and is used to execute the image recognition method provided by the present application.

[0069] As an optional implementation, the first terminal 11 may include an image to be identified, image display software, an image output unit GPU (Graphics Processing Unit), and a screen connected via a preset interface (e.g., HDMI (High Definition Multimedia Interface) interface). The image display software is used to display the image to be identified, and the screen is used to display the identified image data in real time.

[0070] As an optional implementation, in the case where the above-mentioned image injection module 12 is a video injection chassis, the video injection chassis may include a multi-channel (for example, four-channel) video injection board, an Ethernet interface and a power interface provided by the chassis.

[0071] As an optional implementation, the controller 13 may include a camera input module, a power module and an Ethernet module. The camera input module may be used to receive image data sent by the image injection module 12, and the Ethernet module may be used to establish a network connection with the second terminal 14.

[0072] As an optional implementation method, the second terminal 14 may include multiple Ethernet interfaces (for example, three, which can be connected to the first terminal 11, the image injection module 12, and the controller 13 respectively), a GPU, a power module, and a screen connected via an HDMI interface, and the screen is used to display the image data recognized by the controller in real time.

[0073] Based on the above structure, the detailed connection between the first terminal 11, the image injection module 12, the controller 13, and the second terminal 14 can be referred to Figure 2 Shown: See Figure 2 , is a structural diagram of another image recognition system provided in an embodiment of the present application. Figure 2 The structure shown is illustrated by taking the first terminal 11 as PC1, the image injection module 12 as the video injection chassis, the controller 13 as ECU, and the second terminal 12 as PC2 as an example. Figure 2 As shown, the image recognition system 10 may include: PC1, video injection chassis and ECU (intelligent driving controller), PC2.

[0074] Among them, PC1 includes red, green and blue card images, color card display software and image output unit GPU, as well as a screen connected to PC1 via HDMI. The above screen mainly displays the red, green and blue card information in real time; the video injection chassis includes a four-channel video injection board, as well as the chassis's own Ethernet interface and power interface; ECU includes a camera input module, a power module and an Ethernet module; PC2 includes three Ethernet interfaces, a GPU and a power module, as well as a screen connected to PC2 via HDMI. The function of the above screen is mainly to display the color card information recognized by the ECU in real time.

[0075] Among them, PC1 is connected to the video injection board input board of the video injection box through the HDMI port of the GPU, the output channel 1 of the video injection board is connected to the input end of the camera module of the ECU through the LVDS harness, PC2 is connected to the ECU, video injection box and PC1 through Eth1, Eth2 and Eth3 respectively to realize data interaction, and power modules are deployed in PC1, video injection chassis, ECU and PC2 respectively, among which PC1, PC2 and video injection chassis are configured with 22V input, and ECU is configured with 12V.

[0076] Based on the above system structure, Figure 2The implementation process of the image recognition system shown is as follows: PC1 starts and opens the prepared red, green, and blue standard color cards. Then, the image data of the color cards is output through the HDMI interface of the GPU, transmitted to the video injection chassis, and enters the video injection system through input channel 1 of the video injection board. The video injection board processes the image data and then transmits the processed image data to the camera input end of the intelligent driving controller (ECU) through the LVDS cable harness. After receiving the LVDS data, the controller further processes the data and applies the processing results to each module inside the controller.

[0077] Based on this, PC2 can execute the image recognition method provided in this application to solve the technical problem that when the camera parameters change or the image recognition program inside the intelligent driving controller is upgraded, the current image injection device cannot meet the changed usage requirements, resulting in a large difference between the image recognized by the controller and the image input by the image injection device, affecting the user experience. Thus, it realizes calibrating the parameters of the image injection module in the online calibration image recognition system, improves the accuracy of the controller's image recognition, and enhances the user experience.

[0078] The following further explains the image recognition method provided in this application with specific examples in conjunction with the accompanying drawings. The examples do not constitute a limitation to the embodiments of this application.

[0079] See Figure 3 , which is the flowchart of an embodiment of an image recognition method provided in an embodiment of this application. As an embodiment, the image recognition method provided in an embodiment of this application can be applied to the second terminal in the image recognition system. Among them, the image recognition system can include a first terminal, an image injection module, a controller, and a second terminal. The above second terminal can be respectively connected to the first terminal, the image injection module, and the controller. The above first terminal can be connected to the image injection module. The above image injection module can be connected to the controller. The image recognition system and its composition structure can refer to Figure 1 the image recognition system 10 shown. As Figure 3 shown, the process can include the following steps:

[0080] Step 301, obtain the first image data of the first terminal for recognizing the image to be recognized and the second image data of the controller for recognizing the image to be recognized; wherein, the first terminal sends the first image data to the image injection module for processing to obtain the third image data, and the image injection module can send the third image data to the controller for processing to obtain the second image data.

[0081] The above first terminal refers to the terminal in the image recognition system that stores the image to be recognized and recognizes the image to be recognized. For example Figure 1 the first terminal 11 shown.

[0082] The above-mentioned image to be recognized refers to the image to be recognized stored in the first terminal. The image to be recognized can be an image collected during the actual driving of the vehicle. In the autonomous driving simulation test environment, in order to improve the recognition accuracy, the image to be recognized can also be a standard red, green, and blue color card image.

[0083] The above-mentioned first image data refers to the image data obtained after the first terminal recognizes the image to be recognized. It can be the RGB (Red, Green, Blue) pixel values of the image to be recognized, or other image data. The embodiments of the present application do not limit this.

[0084] The above-mentioned controller refers to a control unit used to recognize the image to be recognized and apply the recognized image data. It can be an intelligent driving controller in the vehicle, or other controllers. The embodiments of the present application do not limit this.

[0085] The above-mentioned second image data refers to the image data obtained by the controller further recognizing the image to be recognized. This image data can be the RGB pixel data of the image to be recognized, or other image data. The embodiments of the present application do not limit this.

[0086] In some embodiments of the present application, during the process of recognizing the image to be recognized by the image recognition system, the above-mentioned image to be recognized can be stored in the first terminal. Based on this, the first terminal can first recognize the image to be recognized to obtain corresponding image data (for the sake of distinction, hereinafter referred to as "first image data").

[0087] As an alternative implementation, the first terminal can read the pixel values of the image to be recognized through a preset pixel reading tool and determine the read pixel values as the first image data.

[0088] After that, the first terminal can send the above-mentioned first image data to the image injection module. After receiving the first image data sent by the first terminal, the image injection module can further process the first image data to obtain image data (for the sake of description, hereinafter referred to as "third image data"). Among them, the processing of the first image data by the image injection module can include but is not limited to: configuring the image attributes of the image corresponding to the first image data (configuration parameters include but are not limited to: parameters such as width, height, frame rate, format, and color space), setting the embedded line attributes (such as the number of rows at the head of the image, the number of rows at the tail of the image, and the number of independent embedded rows), and reverse ISP (Image Signal Processor) color adjustment and other processing.

[0089] After that, after obtaining the third image data, the image injection module can inject the third image data into the controller. After receiving the third image data, the controller can further process the third image data to obtain the processed image data (hereinafter referred to as "the second image data for convenience of description"), and apply the second image data to each module inside the controller. Among them, the processing of the third image data by the controller can be forward ISP color correction processing, and can also include other processing, which is not limited in the embodiments of the present application.

[0090] Based on the above processing flow, in order to ensure the accuracy of the controller's recognition of the image to be recognized, the second terminal can respectively obtain the first image data of the first terminal's recognition of the image to be recognized and the second image data of the controller's recognition of the image to be recognized in the above processing flow, so as to determine the accuracy of the controller's recognition of the image to be recognized according to the above first image data and second image data.

[0091] As for how the second terminal specifically obtains the above first image data and second image data, it will be described in the following through Figure 6 the process shown, which will not be elaborated here.

[0092] Step 302, in the case where it is determined that the above second image data and the first image data do not meet the preset conditions, adjust the parameters of the image injection module so that the above first image data and the above second image data meet the preset conditions.

[0093] The above preset conditions refer to the conditions preset for characterizing that the second image data of the image to be recognized recognized by the controller is relatively accurate. In the embodiments of the present application, the preset conditions can be that the first image data and the second image data are the same or the difference is less than the preset value.

[0094] The parameters of the above image injection module refer to the relevant parameters used to process the image to be recognized in the image injection module, which can include but are not limited to: image attribute parameters (including but not limited to: parameters such as width, height, frame rate, format, and color space), embedded row attribute parameters (such as the number of image header rows, the number of image tail rows, and the number of independent embedded rows), whether to enable reverse ISP color correction, and in the case of enabling reverse ISP color correction, the matrix parameters related to reverse ISP color correction.

[0095] In some embodiments of the present application, in order to determine the accuracy of the controller's processing of the image to be recognized, after the second terminal obtains the first image data of the first terminal's recognition of the image to be recognized and the second image data of the controller's recognition of the image to be recognized, the second image data and the first image data are compared.

[0096] Among them, since the first image data is the image data of the image to be recognized directly read by the first terminal, and it is the image data of the image to be recognized with relatively high accuracy. Therefore, by comparing the second image data with the first image data, it can be determined whether the second image data recognized by the controller is relatively accurate.

[0097] Optionally, when it is determined that the second image data and the first image data meet the preset conditions after comparison, it can be determined that the second image data is relatively close to or the same as the first image data. Therefore, it can be determined that the second image data recognized by the controller is relatively accurate, and the parameters of the image injection module can be not adjusted.

[0098] Optionally, when it is determined that the second image data and the first image data do not meet the preset conditions after comparison, it can be determined that the difference between the second image data and the first image data is relatively large, indicating that the second image data recognized by the controller is inaccurate. Therefore, the parameters of the image injection model can be adjusted to make the first image data and the second image data meet the preset conditions. Here, the first image data and the second image data meeting the preset conditions means that after the parameters of the image injection module are adjusted, the first terminal, the image injection module, and the controller re-recognize the object to be recognized, the second terminal re-obtains the first image data after the first terminal recognizes the image to be recognized, and re-obtains the third image data after the controller recognizes the image to be recognized, and it is determined that the re-obtained first image data and the third image data meet the preset conditions.

[0099] As for how to specifically adjust the parameters of the image injection module, it will be described below and will not be elaborated here first.

[0100] The technical solution provided by the embodiments of the present application obtains the first image data obtained by the first terminal for recognizing the image to be recognized, and the second image data obtained by the controller for recognizing the image to be recognized. Among them, the first terminal sends the first image data to the image injection module for processing to obtain the third image data, and the image injection module can send the third image data to the controller for processing to obtain the second image data. When it is determined that the second image data and the first image data do not meet the preset conditions, the parameters of the image injection module are adjusted so that the first image data and the second image data meet the preset conditions. In this technical solution, during the process of image recognition by the image recognition system, the second control terminal respectively obtains the image data obtained by the first terminal and the controller for recognizing the image to be recognized, and when it is determined that the image data recognized by the first terminal and the controller for the image to be recognized is inconsistent, the parameters of the image injection module can be adjusted online to make the image data recognized by the controller consistent with the image data recognized by the first terminal, ensuring the accuracy of the controller's recognition, realizing the online calibration of the parameters of the image injection module in the image recognition system, improving the accuracy of the controller's image recognition, and enhancing the user experience.

[0101] See Figure 4 , which is a flowchart of an embodiment of another image recognition method provided by the embodiments of the present application. Figure 4 The process shown Figure 3 On the basis of the process shown, it describes how to determine that the first image data and the third image data meet the preset conditions after adjusting the parameters of the image injection module. As Figure 4 shown, the process may include the following steps:

[0102] Step 401, obtain the first image data obtained by the first terminal for recognizing the image to be recognized, and the second image data obtained by the controller for recognizing the image to be recognized.

[0103] Step 402, determine whether the first image data and the second image data meet the preset conditions. If so, end the process; if not, execute step 403.

[0104] Step 403, adjust the parameters of the image injection module.

[0105] For the detailed description of steps 401 to 403, reference can be made to the description in steps 301 and 302, which will not be elaborated here.

[0106] Step 404, restart the image injection module.

[0107] Step 405, re-obtain the first image data obtained by the first terminal for recognizing the image to be recognized, and the second image data obtained by the controller for recognizing the image to be recognized.

[0108] Step 406: Determine whether the newly obtained first image data and second image data meet the preset conditions. If so, end the process; if not, return to step 403 and execute it again.

[0109] The following is a unified description of steps 404 to 406:

[0110] In some embodiments of the present application, after the second terminal completes the parameter adjustment of the image injection module, in order to make the adjusted parameters take effect, the image injection module can be restarted.

[0111] Based on this, after the image recognition system detects that the image injection module is restarted, it can re-recognize the image to be recognized. That is, the first terminal re-recognizes the image to be recognized to obtain the first image data, and sends the first image data to the image injection module. Then, the image injection module processes the received first image data to obtain the third image data, and sends the third image data to the controller for processing. After receiving the third image data, the controller can process the third image data to obtain the second image data.

[0112] In this regard, the second terminal can newly obtain the first image data obtained by the first terminal for recognizing the image to be recognized, and the second image data obtained by the controller for recognizing the image to be recognized.

[0113] After that, the newly obtained first image data and second image data can be compared to determine whether the first image data and the third image data meet the preset conditions.

[0114] Optionally, if the newly obtained first image data and second image data meet the preset conditions, it means that the controller's recognition of the image to be recognized is relatively accurate at this time. Therefore, the parameter adjustment of the image injection module can be stopped and the process can be ended.

[0115] Optionally, if the newly obtained first image data and second image data do not meet the preset conditions, it means that the controller's recognition of the image to be recognized is inaccurate at this time. Therefore, the process can return to continue the step of adjusting the parameters of the image injection module, that is, re-adjust the parameters of the image injection module until, after adjusting the parameters of the image injection module and restarting the image injection module, the newly obtained first image data and second image data meet the preset conditions.

[0116] In the technical solution provided by the embodiments of the present application, after adjusting the parameters of the image injection module, the image injection module is restarted, and the first image data obtained by the first terminal for recognizing the image to be recognized and the second image data obtained by the controller for recognizing the image to be recognized are acquired again. When it is determined that the first image data and the second image data do not meet the preset conditions, the parameters of the image injection module are adjusted again until the first image data and the second image data acquired again meet the preset conditions. In this technical solution, by adjusting the parameters of the image injection module multiple times until the first image data and the second image data acquired again meet the preset conditions, the accuracy of the online adjustment of the parameters of the image injection module can be ensured, thereby ensuring the accuracy of the controller in recognizing the image to be recognized.

[0117] In some embodiments of the present application, the image to be recognized may be a preset color card image, such as a red, green, and blue color card image. The first image data may be the first pixel value of the color card image, and the second image may be the second pixel value of the color card image.

[0118] As an implementation manner, the first terminal may recognize the pixel values in a preset area of the color card image and determine the recognized pixel values as the first image data. The preset area may be the lower right corner of the color card image. For example, see Figure 5 , which is a schematic diagram of a color card image provided by the embodiments of the present application. As Figure 5 shown, the color card image may include a red color card image, a green color card image, and a blue color card image. The first terminal may obtain the corresponding pixel values by respectively recognizing the lower right corners ( Figure 5 shown as areas P1, P2, and P3) of each color card image.

[0119] Based on this, in the process shown in Figure 3 or Figure 4 above, when determining whether the first image data and the second image data meet the preset conditions, it may be compared whether the first pixel value and the second pixel value are consistent. Among them, by comparing the pixel values recognized by the first terminal and the pixel values recognized by the controller, the accuracy of the controller in recognizing the image to be recognized can be evaluated more accurately.

[0120] Optionally, when it is determined that the first pixel value and the second pixel value are consistent, it may be determined that the first image data and the second image data meet the preset conditions.

[0121] Optionally, when it is determined that the first pixel value and the second pixel value are inconsistent, it may be determined that the first image data and the second image data do not meet the preset conditions.

[0122] See Figure 6 , which is a flowchart of an embodiment of another image recognition method provided by the embodiments of the present application.Figure 5 The process shown in Figure 3 or Figure 4 On the basis of the process shown, it describes how to obtain the first image data of the first terminal for recognizing the image to be recognized and the second image data of the controller for recognizing the image to be recognized. As Figure 5 shown, this process may include the following steps:

[0123] Step 601: Obtain the first image data of the image to be recognized saved in the first terminal through a preset first remote control instruction.

[0124] In some embodiments of the present application, the second terminal may be connected to the first terminal, the image injection module, and the controller respectively through an Ethernet interface (such as Figure 2 the Ethernet interfaces Eth1, Eth2, and Eth3 shown). Therefore, when the second terminal interacts with the first terminal, the image injection module, and the controller respectively, it can interact through remote control instructions.

[0125] Based on this, when the second terminal obtains the first image data of the first terminal, it can obtain the first image data of the image to be recognized saved in the first terminal through a preset first remote control instruction. For example, the second terminal can obtain the first image data of the image to be recognized saved in the first terminal through a preset SSH (Secure Shell) instruction.

[0126] Step 602: Obtain the processed image of the processor through a preset second remote control instruction.

[0127] Step 603: Recognize the above-mentioned processed image to obtain the second image data.

[0128] The following is a unified description of steps 602 and 603:

[0129] The above-mentioned processed image refers to the image recognized by the controller.

[0130] In some embodiments of the present application, when the second terminal obtains the second image data of the image to be recognized processed by the controller, in order to ensure the accuracy of the obtained second image data, it can first obtain the processed image after the controller recognizes the image to be recognized through a second remote control instruction and save the processed image in the form of a picture.

[0131] After that, the processed image can be recognized to obtain the second image data.

[0132] As an alternative implementation, the image to be recognized described above may be a preset color card image, and the first image data and the second image data of the image to be recognized may both be the pixel values of the color card image. Based on this, the second terminal may recognize a preset area of the processed image to obtain the second image data described above.

[0133] The technical solution provided by the embodiments of the present application obtains the first image data of the image to be recognized saved by the first terminal through a preset first remote control instruction, obtains the processed image of the processor through a preset second remote control instruction, and recognizes the processed image to obtain the second image data. In this technical solution, when the second terminal obtains the corresponding image data from the first terminal and the controller respectively, it can be directly obtained through the remote control instruction. The obtaining method is simple and direct, which can improve the efficiency and accuracy of image data acquisition.

[0134] See Figure 7 , which is a flowchart of another embodiment of the image recognition method provided by the embodiments of the present application. Figure 7 The process shown Figure 3 or Figure 4 On the basis of the process shown, it describes how to adjust the parameters of the image injection module specifically. As Figure 7 shown, the process may include the following steps:

[0135] Step 701, configure the network parameters of the image injection module through a remote control instruction.

[0136] The above network parameters refer to the parameters related to the network connection of the image injection module, which may include but are not limited to: IP (Internet Protocol) address and port, etc.

[0137] In some embodiments of the present application, the second terminal may be connected to the first terminal, the image injection module, and the controller respectively through an Ethernet interface (such as Figure 2 the Ethernet interfaces Eth1, Eth2, and Eth3 shown), so when the second terminal interacts with the first terminal, the image injection module, and the controller respectively, it can interact through a remote control instruction.

[0138] Based on this, when the second terminal configures the parameters of the image injection module, it may first configure the network parameters of the image injection module through a remote control instruction so that the second terminal can communicate with the image injection module normally.

[0139] As an alternative implementation, the second terminal can first enter the control system of the image injection module (such as the board control system of the video injection chassis) through a remote control instruction (such as an SSH instruction). After that, the network parameters of the image injection module (such as IP address and port) can be configured through the remote control instruction, so that the second terminal can communicate with the image injection module normally.

[0140] Step 702: When it is determined that normal communication with the image injection module is established, obtain the image parameter configuration for the image injection module.

[0141] Step 703: Configure the image parameters in the image injection module according to the above image parameter configuration.

[0142] The following is a unified description of Step 702 and Step 703:

[0143] The above image parameter configuration refers to the configuration parameters for configuring the image parameters in the image injection module.

[0144] The above image parameters refer to the parameters related to image attributes involved when the image injection module processes the image to be recognized, which may include but are not limited to: image attribute parameters (such as parameters like width, height, frame rate, format, and color space), embedded line attribute parameters (such as the number of image header lines, the number of image tail lines, and the number of independent embedded lines), etc.

[0145] In some embodiments of the present application, after the second terminal completes the network parameter configuration of the image injection module, when it is determined that normal communication with the image injection module can be established currently, the current image parameter configuration for the image injection module can be obtained. After that, the image parameters in the image injection module can be configured according to the above image parameter configuration.

[0146] As an alternative implementation, the above image parameter configuration can be input by the user. Therefore, the second terminal can obtain the current image parameter configuration for the image injection module through the visualization interface.

[0147] As another alternative implementation, the above image parameter configuration can be pre-stored in the storage medium. Therefore, the second terminal can obtain the current image parameter configuration for the image injection module from the preset storage medium.

[0148] As an alternative implementation, the above image parameters may include image attribute parameters (such as parameters like width, height, frame rate, format, and color space) and embedded line attribute parameters (such as the number of image header lines, the number of image tail lines, and the number of independent embedded lines).

[0149] Based on this, the second terminal can first obtain the above image attribute parameter configuration, and configure the image attribute parameters in the image injection module according to the image attribute parameter configuration. Then, it can obtain the above embedded line attribute parameter configuration, and configure the embedded line attribute parameters in the image injection module according to the embedded line attribute parameter configuration.

[0150] Step 704: When it is determined that the color adjustment function is currently enabled, obtain the corresponding adjustment matrix and offset matrix, and input the above adjustment matrix and offset matrix into the image injection module.

[0151] The above color adjustment function refers to the function of the image injection module to adjust the color balance of the image to be recognized and enhance the image quality of the image to be recognized. For example, the ISP color adjustment function. Among them, in the image injection module, the ISP color adjustment function can be an inverse ISP color adjustment function. After the image injection module performs inverse ISP color adjustment processing on the first image data to obtain the third image data, the controller can perform forward ISP color adjustment processing on the third image data sent by the image injection module.

[0152] The above adjustment matrix and offset matrix are both parameter matrices for color adjustment of the image to be recognized.

[0153] In some embodiments of the present application, the user can set whether to enable the ISP color adjustment function of the image injection module through a visualization interface. Based on this, the second terminal can determine whether the ISP color adjustment function of the image injection module is currently enabled through the visualization interface.

[0154] Optionally, if it is determined that the ISP color adjustment function is not currently set to be enabled, the ISP color adjustment function of the image injection module can be directly set to be disabled.

[0155] Optionally, if it is determined that the ISP color adjustment function is currently set to be enabled, the corresponding adjustment matrix and offset matrix of the ISP color adjustment function can be obtained, and the adjustment matrix and offset matrix can be input into the image injection module, so that when the image injection module performs ISP color adjustment on the image to be recognized, it can perform color adjustment on the image to be recognized according to the adjustment matrix and offset matrix.

[0156] As an implementation manner, the above adjustment matrix and offset matrix can be input by the user through a visualization interface. Based on this, the second terminal can obtain the adjustment matrix and offset matrix input by the user through the visualization interface.

[0157] As another implementation manner, the above adjustment matrix and offset matrix can be pre-stored in a preset storage medium. Based on this, the second terminal can obtain the above adjustment matrix and offset matrix from the storage medium.

[0158] In addition, in some embodiments of the present application, when the image injection module enables the ISP color adjustment function, the image injection module can process the received first image data in the following manner to obtain the third image data:

[0159] First, the current adjustment matrix and offset matrix can be obtained. Then, the first image data can be processed according to the above adjustment matrix and offset matrix to obtain the third image data. Through this reverse ISP color adjustment technology, it is possible to accurately adjust the color distribution of the image to be recognized and ensure the accuracy of the image recognized by the controller.

[0160] As an exemplary implementation, the first image data can be processed by the following formula (1) to obtain the third image data:

[0161]

[0162] Where: Rin, Gin, and Bin respectively represent the first image data input from the first terminal to the image injection module, which are the values of the red, green, and blue color components; Rout, Gout, and Bout respectively represent the third image data obtained after the image injection module performs inverse transformation ISP processing on the first image data; Rp, Gp, and Bp are the offsets added to each color component, that is, the values of the offset matrix, used to adjust the brightness or contrast of the color; in the adjustment matrix, Rx, Ry, and Rz are the elements of the first row in the 3x3 adjustment matrix, which are multiplied by the RGB components of the first image data to affect the red component of the output image; Gx, Gy, and Gz are the elements of the second row in the 3x3 matrix, which are multiplied by the RGB components of the first image data to affect the green component of the output image; Bx, By, and Bz are the elements of the third row in the adjustment matrix, which are multiplied by the RGB components of the first image data to affect the blue component of the output image.

[0163] Based on this, after the input first image data is transformed by the adjustment matrix, an accurate mapping of the first image data in the color space can be achieved. The above transformation not only includes basic linear mapping but also more detailed adjustments to color balance, saturation, and contrast. Each element in the adjustment matrix, such as Rx, Ry, and Rz, can be weighted for the red, green, and blue components of the first image data, thereby affecting the corresponding color components of the third image data. In addition, by introducing the offset matrix Rp, Gp, and Bp, the brightness and contrast of the image can be further fine-tuned to achieve more accurate parameter correspondence.

[0164] Optionally, in the field of intelligent driving simulation testing, optimizing the image parameters involved above can significantly improve the accuracy of online calibration of camera simulation. This means that in actual applications, engineers can more quickly construct an accurate camera simulation link, thereby simulating the behavior of cameras in the real world in a simulation environment. This not only speeds up the development and testing processes of intelligent driving systems, but also significantly improves the confidence level of intelligent driving simulation testing by providing more real and reliable simulation data.

[0165] In the technical solution provided by the embodiments of this application, the network parameters of the image injection module are configured through a remote control instruction. When it is determined that normal communication with the image injection module is established, the image parameter configuration for the image injection module is obtained, and the image parameters in the image injection module are configured according to the above image parameter configuration. When it is determined that the color adjustment function is currently enabled, the corresponding adjustment matrix and offset matrix are obtained, and the above adjustment matrix and offset matrix are input into the image injection module. In this technical solution, when the second terminal adjusts the parameters of the image injection module, the network parameters of the image injection module are preferentially configured with a remote control instruction to ensure normal communication with the image injection module, and when it is determined that normal communication with the image injection module is established, the image parameters and ISP color adjustment function of the image injection module are configured, achieving rapid and accurate adjustment of the parameters of the image injection module.

[0166] In some embodiments of this application, the above first terminal may include a first graphics processor, such as a GPU. The first graphics processor may be connected to the image injection module through a first preset interface (such as an HDMI interface). The first graphics processor may be connected to a preset first display module through a second preset interface (such as an HDMI interface). The first display module may be used to display the first image data obtained by the first terminal for recognizing the image to be recognized. The first display module may be a display screen, such as Figure 2 the display shown.

[0167] Optionally, the above second terminal may include a second graphics processor. The second graphics processor may be connected to a preset second display module through a third preset interface (such as an HDMI interface). The second display module may be used to display the first image data and the second image data, as well as the comparison result between the two. The above second display module may be a display screen, such as Figure 2 the display shown.

[0168] See Figure 8 , which is a block diagram of an embodiment of an image recognition device provided by the embodiments of this application. As an embodiment, Figure 8The device shown can be applied to the second terminal in an image recognition system. The image recognition system may include a first terminal, an image injection module, a controller, and a second terminal. The second terminal is respectively connected to the first terminal, the image injection module, and the controller. The first terminal is connected to the image injection module, and the image injection module is connected to the controller. For example Figure 1 the image recognition system 10 shown. As Figure 8 shown, the device may include:

[0169] An acquisition module 81, configured to acquire first image data obtained by the first terminal for recognizing an image to be recognized, and second image data obtained by the controller for recognizing the image to be recognized;

[0170] Wherein, the first terminal sends the first image data to the image injection module for processing to obtain third image data, and the image injection module sends the third image data to the controller for processing to obtain the second image data;

[0171] An adjustment module 82, configured to adjust parameters of the image injection module when it is determined that the second image data and the first image data do not meet a preset condition, so that the first image data and the second image data meet the preset condition.

[0172] As Figure 9 shown, it is a schematic structural diagram of a terminal device provided by an embodiment of the present application, including a processor 91, a communication interface 92, a memory 93, and a communication bus 94. Among them, the processor 91, the communication interface 92, and the memory 93 communicate with each other through the communication bus 94,

[0173] The memory 93 is used to store a computer program;

[0174] In an embodiment of the present application, when the processor 91 executes the program stored on the memory 93, it implements the image recognition method provided by any one of the foregoing method embodiments, including:

[0175] Acquiring first image data obtained by the first terminal for recognizing an image to be recognized, and second image data obtained by the controller for recognizing the image to be recognized;

[0176] Wherein, the first terminal sends the first image data to the image injection module for processing to obtain third image data, and the image injection module sends the third image data to the controller for processing to obtain the second image data;

[0177] In the case where it is determined that the second image data and the first image data do not meet the preset conditions, the parameters of the image injection module are adjusted so that the first image data and the second image data meet the preset conditions.

[0178] An embodiment of the present application also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image recognition method provided in any of the foregoing method embodiments are implemented.

[0179] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] Through the above-described embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the related technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0181] It should be understood that the terms used herein are only for the purpose of specific example embodiments of the text and are not intended to be limiting. Unless otherwise clearly indicated in the context, the singular forms "a", "an", and "the" as used herein may also include the plural forms. The terms "include", "comprise", "contain", and "have" are inclusive and thus specify the presence of the stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations in the text are not to be construed as necessarily requiring them to be executed in the specific order stated or illustrated, unless the execution order is clearly indicated. It should also be understood that alternative or additional steps may be used.

[0182] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. An image recognition method, characterized in that: A second terminal applied to an image recognition system, the image recognition system comprising a first terminal, an image injection module, a controller, and a second terminal, the second terminal being connected to the first terminal, the image injection module, and the controller respectively, the first terminal being connected to the image injection module, the image injection module being connected to the controller, the method comprising: Acquire first image data of the image to be recognized by the first terminal, and second image data of the image to be recognized by the controller; The first terminal sends the first image data to the image injection module for processing to obtain third image data, and the image injection module sends the third image data to the controller for processing to obtain the second image data; When it is found by comparison that the second image data and the first image data do not satisfy a preset condition, the parameters of the image injection module are adjusted so that the first image data and the second image data satisfy the preset condition.

2. The method according to claim 1, characterized in that After adjusting the parameters of the image injection module, the method further includes: Restarting the image injection module; Reacquiring first image data of the image to be recognized by the first terminal and second image data of the image to be recognized by the controller; When it is found by comparison that the first image data and the second image data do not satisfy the preset condition, the step of adjusting the parameters of the image injection module is returned to be executed until the first image data and the second image data satisfy the preset condition.

3. The method according to claim 1, characterized in that The acquiring first image data of the image to be recognized by the first terminal and second image data of the image to be recognized by the controller includes: Acquire, by means of a preset first remote control instruction, first image data of the image to be recognized stored in the first terminal; Acquiring a processed image of the controller through a preset second remote control instruction; The processed image is recognized to obtain the second image data.

4. The method according to claim 1, characterized in that: The image to be identified is a preset color card image, the first image data is a first pixel value of the color card image, and the second image data is a second pixel value of the color card image; The second image data and the first image data do not satisfy a preset condition, including: comparing whether the first pixel value and the second pixel value are consistent; When it is found that the first pixel value and the second pixel value are inconsistent after comparison, it is determined that the first image data and the second image data do not satisfy a preset condition.

5. The method according to claim 1, characterized in that The adjusting the parameters of the image injection module includes: Configuring network parameters of the image injection module through remote control instructions; When it is determined that the communication with the image injection module is normal, obtaining image parameter configuration for the image injection module; configuring image parameters in the image injection module according to the image parameter configuration; When it is determined that the color adjustment function is currently turned on, a corresponding adjustment matrix and an offset matrix are obtained, and the adjustment matrix and the offset matrix are input into the image injection module.

6. The method according to claim 5, characterized in that The image injection module processes the received first image data in the following manner to obtain third image data: Get the current adjustment matrix and offset matrix; The first image data is processed according to the adjustment matrix and the offset matrix to obtain the third image data.

7. The method according to claim 1, characterized in that The first terminal includes a first graphics processor, the first graphics processor is connected to the image injection module via a first preset interface, the first graphics processor is connected to a preset first display module via a second preset interface, and the first display module is used to display the first image data; The second terminal includes a second graphics processor, the second graphics processor is connected to a preset second display module via a third preset interface, and the second display module is used to display the first image data and the second image data.

8. An image recognition device, characterized in that: A second terminal used in an image recognition system, the image recognition system comprising a first terminal, an image injection module, a controller, and a second terminal, the second terminal being connected to the first terminal, the image injection module, and the controller respectively, the first terminal being connected to the image injection module, the image injection module being connected to the controller, the device comprising: An acquisition module, used for acquiring first image data of the image to be recognized by the first terminal, and second image data of the image to be recognized by the controller; The first terminal sends the first image data to the image injection module for processing to obtain third image data, and the image injection module sends the third image data to the controller for processing to obtain the second image data; The adjustment module is used to adjust the parameters of the image injection module when it is compared that the second image data and the first image data do not meet the preset conditions, so that the first image data and the second image data meet the preset conditions.

9. An image recognition system, characterized in that: include: A first terminal, an image injection module, a controller, and a second terminal; The second terminal is connected to the first terminal, the image injection module, and the controller respectively, the first terminal is connected to the image injection module, and the image injection module is connected to the controller; The first terminal is used to store the image to be identified, identify the image to be identified, obtain first image data, and send the first image data to the image injection module; The image injection module is used to receive the first image data, process the first image data to obtain third image data, and send the third image data to the controller; The controller is used to receive the third image data and process the third image data to obtain the second image data; The second terminal is used to execute the image recognition method according to any one of claims 1 to 7.

10. A terminal device, characterized in that: include: A processor and a memory, wherein the processor is used to execute an image recognition program stored in the memory to implement the image recognition method according to any one of claims 1 to 7.

Images