Image recognition method, device and electronic equipment

Abstract

The application relates to an image recognition method and device and electronic equipment, and relates to the technical field of automobiles. The method comprises the following steps: in response to a starting operation of a vehicle, a camera is controlled to capture a road image of a road where the vehicle is located at the current time, and the road image is an image to be corrected. The road image is corrected according to first parameter information and second parameter information of the camera, and a corrected road image is obtained. The first parameter information is parameter information of the camera in a correction position, and the second parameter information is parameter information of the camera in a first position. The first position is the position of the camera on the vehicle at the current time. The setting parameters of the camera in the correction position and the setting parameters of the camera in the first position are deviated, and the setting parameters include an orientation angle and / or position information. The corrected road image is recognized to obtain information of a target object on the road where the vehicle travels.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, in particular to the technical field of automatic driving, and specifically to an image recognition method and device and electronic equipment. BACKGROUND

[0002] Computer vision methods have been widely used in automatic driving and assisted driving systems, and recognition methods using pure vision or point cloud image fusion technology have been widely used in artificial intelligence (AI) based assisted driving and automatic driving systems at all levels.

[0003] For the vision part, due to the differences in parameters such as viewing angle, depth of field, and installation position of the camera installed on the vehicle, the same model of camera on different vehicles will obtain different images when shooting the same object, which increases the difficulty of image recognition. SUMMARY

[0004] One of the purposes of the present application is to provide an image recognition method, device and electronic equipment to reduce the difficulty of image recognition.

[0005] In order to achieve the above purpose, the technical solution adopted by the present application is as follows:

[0006] According to a first aspect of the present application, an image recognition method is provided, applied to a vehicle equipped with a camera. The method comprises: in response to a start operation of the vehicle, controlling the camera to shoot a road image of a road where the vehicle is currently located, the road image being a to-be-corrected image. Correcting the road image according to first parameter information and second parameter information of the camera to obtain a corrected road image. The first parameter information is the parameter information of the camera in a correction position, and the second parameter information is the parameter information of the camera in a first position. The first position is the position of the camera on the vehicle at the current time. The setting parameters of the camera in the correction position and in the first position are different, and the setting parameters include an orientation angle and / or position information. Identifying the corrected road image to obtain information of a target object on the road where the vehicle is driving.

[0007] According to the above technical means, the image recognition method provided by the present application can correct the road image shot by the camera on the vehicle according to the parameter information of the camera in the correction position and the parameter information of the camera on the vehicle at the current time, so as to project the image to the correction position. Due to the deviation between the actual installation position of the camera on the vehicle and the standard position, the shot image is not accurate enough. In the present application, the corrected image is in the correction position. Since the image in the correction position is easier to be recognized, the difficulty of image recognition is reduced.

[0008] In a possible implementation, the parameter information of the camera includes intrinsic parameters and extrinsic parameters. The method further includes: in response to the camera being at a target position on the vehicle, capturing a plurality of images by the camera. The target position includes a calibration position or a first position. Based on the plurality of images, the parameter information of the camera when at the target position is determined. When the target position is the calibration position, the parameter information includes standard intrinsic parameters and standard extrinsic parameters of the camera when at the calibration position. When the target position is the first position, the parameter information includes actual intrinsic parameters and actual extrinsic parameters of the camera when at the first position.

[0009] According to the technical means, the image recognition method can accurately correct the image based on the intrinsic parameters and the extrinsic parameters of the camera.

[0010] In a possible implementation, the "correcting the road image based on the first parameter information and the second parameter information of the camera to obtain a corrected road image" includes: determining a correction coefficient of the camera based on the standard intrinsic parameters and the standard extrinsic parameters and the actual intrinsic parameters and the actual extrinsic parameters of the camera, and correcting the road image based on the correction coefficient to obtain a corrected road image.

[0011] According to the technical means, the image recognition method can accurately determine the correction coefficient of the camera based on the intrinsic parameters and the extrinsic parameters of the camera at the standard position and the intrinsic parameters and the extrinsic parameters of the camera at the first position, because the intrinsic parameters and the extrinsic parameters of the camera at different positions can reflect the offset of the camera. In this way, the image can be quickly corrected based on the correction coefficient.

[0012] In a possible implementation, the correction coefficient of the camera is a ratio between a product of the standard intrinsic parameters and the standard extrinsic parameters of the camera and a product of the actual intrinsic parameters and the actual extrinsic parameters of the camera.

[0013] According to the technical means, the image recognition method can quickly calculate the correction coefficient of the camera.

[0014] In a possible implementation, the "determining the parameter information of the camera when at the target position based on the plurality of images" includes: based on the plurality of images, determining a rotation matrix and a translation vector, a focal length, and coordinates of a center pixel of the plurality of images according to a preset calibration algorithm, obtaining an extrinsic parameter matrix of the camera based on the rotation matrix and the translation vector, and obtaining an intrinsic parameter matrix of the camera based on the focal length and the coordinates of the center pixel of the plurality of images, to obtain the parameter information of the camera.

[0015] According to the above technical means, in the image recognition method provided by the application, the multiple parameters of the camera can be obtained by calibrating the multiple corrected images according to the calibration algorithm, and the intrinsic parameters and extrinsic parameters of the camera can be accurately obtained based on the multiple parameters.

[0016] In a possible implementation, the "identifying the corrected road image to obtain information of a target object on the road on which the vehicle travels" includes: inputting the corrected road image into an image recognition model to obtain information of a target object on the road on which the vehicle travels. The image recognition model has the function of identifying information of an object in an image.

[0017] According to the above technical means, in the image recognition method provided by the application, the information of the object in the image can be quickly recognized based on the image recognition model.

[0018] According to the second aspect provided by the application, an image recognition device is provided, which is applied to a vehicle provided with a camera. The device includes: a control unit configured to control the camera to capture a road image of a road on which the vehicle is located at a current time in response to a starting operation of the vehicle, the road image being a to-be-corrected image; a correction unit configured to correct the road image according to first parameter information and second parameter information of the camera to obtain a corrected road image, the first parameter information being parameter information of the camera in a correction position, and the second parameter information being parameter information of the camera in a first position, the first position being a position of the camera on the vehicle at the current time, and the camera having a deviation in setting parameters between the correction position and the first position, the setting parameters including an orientation angle and / or position information; and an identification unit configured to identify the corrected road image to obtain information of a target object on the road on which the vehicle travels.

[0019] In a possible implementation, the parameter information of the camera includes intrinsic parameters and extrinsic parameters. The device further includes a determination unit configured to capture multiple images by the camera when the camera is in a target position on the vehicle in response to the camera being in the target position. The target position includes the correction position or the first position. The parameter information of the camera when the camera is in the target position is determined based on the multiple images. When the target position is the correction position, the parameter information includes standard intrinsic parameters and standard extrinsic parameters of the camera when the camera is in the correction position. When the target position is the first position, the parameter information includes actual intrinsic parameters and actual extrinsic parameters of the camera when the camera is in the first position.

[0020] In a possible implementation, the correction unit is specifically configured to determine a correction coefficient of the camera according to the standard intrinsic parameters and the standard extrinsic parameters of the camera and the actual intrinsic parameters and the actual extrinsic parameters of the camera, and correct the road image according to the correction coefficient to obtain the corrected road image.

[0021] In a possible implementation, the correction coefficient of the camera is a ratio between a product of a standard intrinsic parameter and a standard extrinsic parameter of the camera and a product of an actual intrinsic parameter and an actual extrinsic parameter of the camera.

[0022] In a possible implementation, the determining unit is specifically configured to determine, based on the plurality of images, a rotation matrix and a translation vector of the camera, a focal length, and coordinates of a center pixel of the plurality of images according to a preset calibration algorithm, obtain an extrinsic parameter matrix of the camera according to the rotation matrix and the translation vector of the camera, and obtain an intrinsic parameter matrix of the camera according to the focal length and the coordinates of the center pixel of the plurality of images, to obtain the parameter information of the camera.

[0023] In a possible implementation, the recognizing unit is specifically configured to input the corrected road image into an image recognition model to obtain information of a target object on a road on which the vehicle travels. The image recognition model has a function of recognizing information of an object in an image.

[0024] According to a third aspect provided in the present application, an electronic device is provided, which is applied to a vehicle provided with a camera. The electronic device comprises a memory and a processor, which are coupled; the memory is configured to store computer program code, and the computer program code comprises computer instructions; when the processor executes the computer instructions, the electronic device performs the image recognition method provided in the first aspect and any possible implementation thereof.

[0025] According to a fourth aspect provided in the present application, a computer readable storage medium is provided, and the computer readable storage medium stores instructions; when the instructions run on an electronic device, the electronic device performs the image recognition method provided in the first aspect and any possible implementation thereof.

[0026] According to a fifth aspect provided in the present application, a vehicle is provided, which comprises the electronic device provided in the third aspect.

[0027] According to a sixth aspect provided in the present application, a computer program product is provided, and the computer program product comprises computer instructions; when the computer instructions run on an electronic device, the electronic device performs the image recognition method provided in the first aspect and any possible implementation thereof.

[0028] Therefore, the above technical features of the present application have the following beneficial effects:

[0029] (1) After obtaining the road image captured by the camera on the vehicle, the road image can be corrected according to the parameter information of the camera in the correction position and the parameter information of the camera of the vehicle at the current time, so as to project the image to the correction position. Due to the deviation between the actual installation position of the camera of the vehicle and the standard position, the captured image is not accurate enough. In the present application, the corrected image is in the correction position. Since the image in the correction position is easier to be recognized, the difficulty of image recognition is reduced.

[0030] (2) Based on the intrinsic and extrinsic parameters of the camera in different positions, the correction coefficient of the camera is accurately determined, and the image is corrected based on the correction coefficient.

[0031] (3) Based on the corrected image, the image can be accurately detected to determine the information of the objects around the vehicle.

[0032] It should be noted that the technical effects brought by any one of the implementation manners of the second aspect to the sixth aspect can be referred to the technical effects brought by the corresponding implementation manners in the first aspect, which will not be repeated here.

[0033] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a structural schematic diagram of a vehicle according to an exemplary embodiment;

[0035] Figure 2 is a flow chart of an image recognition method according to an exemplary embodiment;

[0036] Figure 3 is a flow chart of another image recognition method according to an exemplary embodiment;

[0037] Figure 4 is a block diagram of an image recognition device according to an exemplary embodiment;

[0038] Figure 5 is a block diagram of an image recognition device according to an exemplary embodiment. DETAILED DESCRIPTION

[0039] The above and other advantages and features of the application will become apparent from the following description of the preferred embodiments, when taken in conjunction with the accompanying drawings. The application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these preferred embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0040] It is noted that the drawings of the embodiments provided in the following description are for the purpose of illustrating the general architecture and concepts of the present application and are not intended to limit the present application. The drawings provided are in simplified form and are not drawn to precise scale. Additionally, the drawings are not intended to be binding in any way and are provided simply to illustrate the concepts underlying the application. The description is not intended to be limited to the embodiments described herein, but rather just one construction thereof. Additionally, the same reference numerals in different drawings represent the same element.

[0041] In the description of the embodiments, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this document is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, "at least one" and "multiple" mean two or more. "First", "second", and the like do not limit the quantity and execution order, and "first", "second", and the like do not necessarily mean different.

[0042] The recognition method of pure vision or point cloud image fusion technology has been widely used in automatic driving technology. The point cloud image based on laser radar method can be obtained based on laser radar method. The point cloud image based on laser radar method is mainly due to the high hardware cost and reliability problem of laser radar itself, which leads to the difficulty of landing, and the single point cloud data lacks the semantic information of image vision, which has difficulty in classifying small objects and targets at a distance.

[0043] Generally, the pure vision of the vehicle and the vision or laser fusion method of each automatic driving solution provider are mainly used. For the vision part, due to the differences in focal length, viewing angle, depth of field, and installation position of each camera installed on the vehicle, different cameras will shoot different images of the same object. Such data is not useful for the training of subsequent recognition models (such as neural network models), so before model training, the camera needs to be standardized to an artificially synthesized virtual camera. Such a unified parameter image input not only improves the accuracy of subsequent vision model training, but also reduces the training time.

[0044] Based on the virtual camera technology, the user can view the virtual object from various angles without affecting the user's operation experience. For example, based on the virtual camera technology, the multimedia data (such as video) can improve the switching fluency of the picture of the multimedia data.

[0045] For virtual cameras in the field of visual perception of automatic driving, the virtual camera technology has not been widely applied and developed.

[0046] In the visual-based automatic driving task, the deep learning method can better detect and identify the target. Therefore, in addition to the internal factors such as distortion of the camera itself, the unified installation position of the camera can also provide stability and reliability for deep learning.

[0047] In an example, during the installation of the camera, due to the error of manual or machine, deviation may occur between the actual installation position of the camera and the standard position. For example, the camera of the vehicle has a certain selection or translation relative to the standard position. In this way, if the image captured by the camera with deviation is used for model training, the accuracy of the recognition result of the trained model may be reduced. In order to increase the accuracy, the image can be calibrated by manual annotation, but when the image is large, the manual and time cost is high.

[0048] For example, in the automatic driving visual imaging system, the vehicle's own coordinate system, the camera coordinate system, and the pixel coordinate system are involved. The own coordinate system generally takes the center of the rear axle of the vehicle as the origin to establish a right-handed coordinate system. In the right-handed coordinate system, the x-axis points forward, the y-axis points left, and the z-axis points upward. The camera coordinate system takes the aperture of the camera as the origin, the x-axis points right, and the y-axis points down. The pixel coordinate system takes the top left corner of the image as the origin, the x-axis points right, and the y-axis points down. In the embodiments of the present application, forward means towards the front of the vehicle, and downward means towards the ground. The same object can be represented by different coordinates in the above-mentioned multiple coordinate systems. Therefore, any angle and installation position deviation of the camera will cause the final imaging to change.

[0049] In view of this, the embodiments of the present application provide a virtual camera in visual-based automatic driving. The installation positions of the cameras on the same vehicle type should be exactly the same, but in actual situations, there will be errors in the angle of orientation and position offset. The technical scheme provided by the embodiments of the present application can project the cameras at the corresponding positions on the same vehicle type to the virtual camera position, which has the advantages of low power consumption, real-time performance, and reliability.

[0050] In an example, to realize the projection of the camera at the corresponding position on the same vehicle model to the virtual camera position, an embodiment of the present application provides a method based on photographic transformation, which converts the image captured by the camera to the image captured by the virtual camera position. Based on the method, it is simple and easy to deploy. At the same time, it does not increase the amount of calculation and time delay. It also does not need manual annotation, which simplifies the model training.

[0051] Figure 1 A vehicle is shown. As shown in Figure 1 , the vehicle can be installed with a plurality of cameras (only 6 are shown in Figure 1 , and more can also be included). For example, the plurality of cameras can be installed at different positions of the vehicle (as shown in Figure 1 , respectively installed at the front end, rear end, left side, right side, etc. of the vehicle). The cameras at different positions can capture images of different directions of the vehicle.

[0052] In an application scenario, for the same vehicle model, the installation positions of the cameras on the vehicle are the same (the position can be referred to as a standard position). Then, due to human or machine errors, the cameras of different vehicles may deviate from the positions or the shooting angles.

[0053] In some embodiments, the vehicle can also be deployed with an electronic device, which can be in communication connection with the plurality of cameras. For example, it can be connected through a controller area network (CAN) bus. The electronic device can be used to identify the images captured by the plurality of cameras to obtain the information of the road where the vehicle is located (such as the number of surrounding vehicles, road condition information, etc.).

[0054] In some embodiments, the vehicle can also include other devices or systems, for example, an automatic driving system, which can control the running state of the vehicle based on the identification result of the electronic device.

[0055] Figure 2 is a flow diagram of an image recognition method according to some exemplary embodiments. In some embodiments, the above-mentioned image recognition method can be applied to the vehicle or the electronic device in the vehicle as shown in Figure 1 . Hereinafter, the embodiments of the present application take the image recognition method applied to the vehicle as an example to describe the above-mentioned image recognition method.

[0056] As shown in Figure 2 , the image recognition method provided by the embodiments of the present application includes the following S201-S203.

[0057] S201, in response to the starting operation of the vehicle, controlling the camera to capture the road image of the road where the vehicle is located at the current time.

[0058] The vehicle can be any of the vehicles in Figure 1 The camera can be any of the cameras in the vehicle shown in Figure 1

[0059] In an example, the camera can be triggered to capture images around the vehicle in response to an operation of starting the vehicle by the driver. In this way, the vehicle can periodically capture road images on a road on which the vehicle travels during driving.

[0060] S202. Correct the road image according to the first parameter information and the second parameter information of the camera, to obtain a corrected road image.

[0061] The first parameter information can be parameter information of the camera of the vehicle in a correction position, or can be parameter information of the camera of the vehicle in a standard position. The second parameter information can be parameter information of the camera of the vehicle in a first position. The first position can be an actual installation position of the camera on the vehicle. For example, the parameter information of the vehicle can include intrinsic parameters (or an intrinsic matrix) and extrinsic parameters (or an extrinsic matrix). The setting parameters of the camera in the correction position and the setting parameters of the camera in the first position are deviated. The setting parameters include an orientation angle of the camera and / or position information on the vehicle.

[0062] In a possible implementation, the vehicle can calibrate the camera in the correction position and the camera in the first position respectively, to obtain the parameter information of the camera in the correction position and the parameter information of the camera in the first position.

[0063] In an application scenario, before the vehicle is shipped, the vehicle can capture a plurality of images through the camera in response to a test operation. Based on the one or more images, the vehicle can determine the parameter information of the camera in a target position. The test operation can be an operation of controlling the camera of the vehicle to capture images. For example, the test operation can be an operation of starting the vehicle by a tester.

[0064] In another example, the vehicle can capture a plurality of images through the camera in response to adjusting the camera to the correction position. Based on the one or more images, the vehicle can determine the parameter information of the camera in the correction position.

[0065] In a possible implementation, the vehicle can calibrate the plurality of images according to a preset calibration algorithm, to determine a rotation matrix, a translation vector, a focal length, and a coordinate of a center pixel of the image of the camera.

[0066] ​The calibration algorithm can be set as needed, for example, can include Zhang Zhengyou calibration method, or Zhang Zhengyou calibration method optimized based on Levenberg-Marquardt (L-M) algorithm.

[0067] In an example, the vehicle can obtain the extrinsic matrix of the camera according to the rotation matrix and the translation vector of the camera, and obtain the intrinsic matrix of the camera according to the toe of the camera and the coordinates of the center pixel of the image.

[0068] The calibration method based on the Zhang Zhengyou calibration method optimized based on the L-M algorithm is described below.

[0069] 1. Obtain the checkerboard of the Zhang Zhengyou calibration method, and control the camera to capture the checkerboard from different angles to obtain a group of images.

[0070] 2. Detect the feature points (such as corner points) of each frame of image to obtain the coordinates of the pixels of the image in the pixel coordinate system, and calculate the coordinates of the pixels of the image in the vehicle's own coordinate system based on the conversion relationship between the coordinate system of the image and the vehicle's own coordinate system.

[0071] 3. Based on the coordinates of the pixels of the image in the pixel coordinate system and the coordinates of the pixels of the image in the vehicle's own coordinate system, determine the focal length of the camera and the coordinates of the center pixel of the image, the rotation matrix and the translation vector.

[0072] Specifically, the method for determining the focal length and the coordinates of the center pixel of the camera, the rotation matrix, the translation vector, can refer to the prior art, and will not be described here.

[0073] In an example, the intrinsic matrix of the camera can be Wherein, f x , f y represents the focal length of the camera, c x , c y represents the center pixel or the translation of the pixel of the image.

[0074] In another example, the extrinsic matrix of the camera can be Wherein, R is the rotation matrix, and t is the translation vector.

[0075] Further, the camera can obtain the radial distortion parameter. The radial distortion parameter can be used for further correction of the image. That is, after the image is corrected according to the parameter information of the camera at different positions, the radial distortion parameter can be used to further correct the corrected image, so that the corrected image is more accurate.

[0076] Similarly, based on the calibration method described above, the vehicle can obtain the parameter information of the camera in the corrected position and in the first position.

[0077] Further, after obtaining the parameter information described above, the vehicle can calculate a correction coefficient according to the parameter information of the camera in the corrected position and in the first position.

[0078] In an example, the correction coefficient can be a ratio of a first value and a second value. The first value can be a product of the intrinsic parameter and the extrinsic parameter of the camera in the corrected position. The second value can be a product of the intrinsic parameter and the extrinsic parameter of the camera in the first position.

[0079] For example, the correction coefficient can be KsPs / KP. Wherein Ks is the intrinsic parameter matrix of the camera in the corrected position, Ps is the extrinsic parameter matrix of the camera in the corrected position, K is the intrinsic parameter matrix of the camera in the first position, and P is the extrinsic parameter matrix of the camera in the first position.

[0080] Further, since the rotation matrix of the camera is fixed, the above correction system can also be simplified as KsTs / KT. Wherein Ts is the translation vector of the camera in the corrected position, and T is the translation vector of the camera in the first position.

[0081] In combination with the above correction coefficient, the corrected road image can be KsTs / KT*S. Wherein S is the road image or road image data in S201.

[0082] S203, identifying the corrected road image to obtain information of a target object on the road on which the vehicle travels.

[0083] Wherein, the information of the target object on the road can be the information of the vehicle, pedestrian, sign, landmark, etc. on the road.

[0084] In an example, the vehicle can input the corrected road image into an image recognition model to obtain the information of the target object on the road on which the vehicle travels.

[0085] Wherein, the image recognition model can have the function of identifying objects in the image. For example, the image recognition model can be the neural network model described above. The image recognition model can be obtained by training images captured by multiple cameras in the corrected position according to a preset algorithm. The preset algorithm can be a neural network algorithm, etc. without limitation.

[0086] It should be noted that in the embodiments of the present application, the distance from the camera in the calibrated position to the target object is the same as the distance from the camera in the first position to the target object.

[0087] Based on Figure 2In the embodiments of the present application, the scheme provided by the embodiments of the present application can correct the road image photographed by the camera on the vehicle according to the parameter information of the camera in the correction position and the parameter information of the camera of the vehicle at the current time, so as to project the image to the correction position. Due to the deviation between the actual installation position of the camera of the vehicle and the standard position, the photographed image is not accurate enough. In the present application, the corrected image is in the correction position. Since the image in the correction position is easier to be recognized, the difficulty of image recognition is reduced.

[0088] In some embodiments, as shown in Figure 3 The method provided by the embodiments of the present application can further include S301.

[0089] S301, using the corrected road image and the corresponding recognition result to iteratively train the image recognition model to obtain the trained image recognition model.

[0090] The recognition result of the road image can include a target object included in the road image and information of the target object.

[0091] In an example, the vehicle can use the preset algorithm and the corrected road image and the recognition result to retrain the image recognition model to obtain the trained image recognition model.

[0092] Based on this, since the recognition result of the road image can reflect the road information of the road at the current time, the model trained by the corrected image and the corresponding recognition result can more accurately perform image recognition, and the accuracy of the image recognition model is improved.

[0093] In some embodiments, the method provided by the embodiments of the present application can further include: obtaining third parameter information of the camera of the vehicle at a second time, and correcting the image photographed by the camera at a third time according to the first parameter information of the camera and the third parameter information.

[0094] The second time is after the first time, and the third time is after the second time.

[0095] In a scenario, as the driving time of the vehicle exceeds the preset time or the driving distance exceeds the preset kilometers, the camera can be affected by external factors (such as accidental collision, external force twisting), so that the orientation or position of the camera is offset again. In this case, the vehicle can respond to the recalibration operation to determine the parameter information of the camera again. Specifically, the calibration method can be referred to, and details are not described herein.

[0096] Further, the vehicle can periodically or in response to a recalibration operation of the tester, recalibrate the camera of the vehicle, and correct the image according to the recalibrated parameter information.

[0097] Based on the embodiment, the vehicle can recalibrate the parameter information of the camera, so that the parameters of the camera are more accurate.

[0098] The above mainly introduces the scheme provided by the embodiments of the present application from the perspective of the method. In order to realize the above functions, the image recognition device includes the hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed in the present application, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0099] The embodiments of the present application can divide the functional modules of the image recognition device according to the above method, for example, the image recognition device can include each functional module corresponding to each functional division, or two or more functions can be integrated in one processing module. The integrated module can be realized in the form of hardware or software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division method.

[0100] Figure 4 A structural schematic diagram of an image recognition device provided by an embodiment of the present application. The image recognition device is used to execute the above image recognition method. As shown in the figure, Figure 4 The image recognition device 40 includes a control unit 401, a correction unit 402, and a recognition unit 403.

[0101] The control unit 401 is configured to control the camera to capture a road image of a road where the vehicle is located at the current time in response to a start operation of the vehicle. The road image is a to-be-corrected image.

[0102] The correction unit 402 is used to correct the road image based on the first parameter information and the second parameter information of the camera to obtain a corrected road image. The first parameter information is the parameter information of the camera at the correction position, and the second parameter information is the parameter information of the camera at the first position. The first position is the current position of the camera on the vehicle. The setting parameters of the camera at the correction position deviate from the setting parameters at the first position, and the setting parameters include orientation angle and / or position information.

[0103] The recognition unit 403 is used to recognize the corrected road image and obtain information about the target objects on the road where the vehicle is traveling.

[0104] Optionally, the camera's parameter information includes intrinsic and extrinsic parameters. For example... Figure 4 As shown, the device further includes a determining unit 404, configured to capture multiple frames of images via the camera in response to the camera being at a target position on the vehicle. The target position may be a calibrated position or a first position. Based on the multiple frames of images, parameter information of the camera at the target position is determined. When the target position is a calibrated position, the parameter information includes standard intrinsic and extrinsic parameters of the camera at the calibrated position. When the target position is the first position, the parameter information includes actual intrinsic and extrinsic parameters of the camera at the first position.

[0105] Optionally, the correction unit 402 is specifically used to determine the correction coefficient of the camera based on the standard intrinsic and standard extrinsic parameters of the camera, as well as the actual intrinsic and actual extrinsic parameters, and to correct the road image based on the correction coefficient to obtain the corrected road image.

[0106] Optionally, the camera's correction factor is the ratio between the product of the camera's standard intrinsic parameters and standard extrinsic parameters and the product of the camera's actual intrinsic parameters and actual extrinsic parameters.

[0107] Optionally, the determining unit 404 is specifically used to determine the camera's rotation matrix, translation vector, focal length, and coordinates of the center pixels of the multi-frame images based on a preset calibration algorithm, and to obtain the camera's extrinsic parameter matrix based on the camera's rotation matrix and translation vector, and to obtain the camera's intrinsic parameter matrix based on the camera's focal length and coordinates of the center pixels of the multi-frame images, so as to obtain the camera's parameter information.

[0108] Optionally, the recognition unit 403 is specifically used to input the corrected road image into the image recognition model to obtain information about target objects on the road where the vehicle is traveling. The image recognition model has the function of recognizing information about objects in the image.

[0109] Figure 5 This is a block diagram illustrating an image recognition device 50 according to an exemplary embodiment.Figure 5 As shown, the image recognition apparatus 50 includes but is not limited to a processor 501 and a memory 502.

[0110] The memory 502 is configured to store executable instructions of the processor 501. It can be understood that the processor 501 is configured to execute the instructions to implement the image recognition method in the above embodiments.

[0111] It should be noted that those skilled in the art can understand that the structure of the image recognition apparatus shown in the above embodiments does not constitute a limitation on the image recognition apparatus, and the image recognition apparatus can include more or fewer components than those shown in the above embodiments, or combine some components, or different component arrangements. Figure 5 It should be noted that those skilled in the art can understand that the structure of the image recognition apparatus shown in the above embodiments does not constitute a limitation on the image recognition apparatus, and the image recognition apparatus can include more or fewer components than those shown in the above embodiments, or combine some components, or different component arrangements. Figure 5 It should be noted that those skilled in the art can understand that the structure of the image recognition apparatus shown in the above embodiments does not constitute a limitation on the image recognition apparatus, and the image recognition apparatus can include more or fewer components than those shown in the above embodiments, or combine some components, or different component arrangements.

[0112] The processor 501 is the control center of the image recognition apparatus, and connects all parts of the image recognition apparatus through various interfaces and lines. The processor 501 executes software programs and / or modules stored in the memory 502 and calls data stored in the memory 502 to perform various functions of the image recognition apparatus and process data, thereby overall monitoring the image recognition apparatus. The processor 501 can include one or more processing units. Optionally, the processor 501 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the modem processor can also not be integrated into the processor 501.

[0113] The memory 502 can be used to store software programs and various data. The memory 502 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs (such as determination units, processing units, etc.) required by at least one function module, etc. In addition, the memory 502 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0114] In the exemplary embodiments, a computer readable storage medium including instructions is also provided, for example, the memory 502 including instructions, which can be executed by the processor 501 of the image recognition apparatus 50 to implement the image recognition method in the above embodiments.

[0115] In actual implementation, Figure 4 The functions of the control unit 401, the correction unit 402, and the recognition unit 403 in the above embodiments can be implemented by the processor 501. Figure 5The processor 501 in the image recognition device calls the computer program stored in the memory 502 to implement. The specific implementation process can refer to the description of the image recognition method in the above embodiment, which will not be repeated here.

[0116] Optionally, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0117] In the example embodiment, the embodiment of the application further provides a vehicle comprising the image recognition device.

[0118] In the example embodiment, the embodiment of the application further provides a computer program product comprising one or more instructions executable by the processor 501 of the image recognition device to complete the image recognition method in the above embodiment.

[0119] It should be noted that the instructions in the above computer readable storage medium or the one or more instructions in the computer program product are executed by the processor of the image recognition device to realize each process of the above image recognition method embodiment, and can achieve the same technical effect as the above image recognition method. To avoid repetition, it will not be repeated here.

[0120] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete the above described full classification part or part of the function.

[0121] In the several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented by other ways. For example, the device embodiment described above is only schematic, for example, the division of the modules or units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection between each other can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separate, and the components displayed as units may be a physical unit or multiple physical units, that is, may be located in one place, or also can be distributed to multiple different places. Part or all of the classified units can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0123] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0124] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the part that contributes to the prior art or the whole classification or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions for making a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk and various program code storage media.

[0125] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image recognition method characterized by, The application is applied to a vehicle which is provided with a camera; the method comprises: In response to a starting operation of the vehicle, a camera is controlled to capture a road image of a road where the vehicle is located at a current time, and the road image is an image to be corrected; In response to the camera being at a target position on the vehicle, a plurality of images are captured by the camera; wherein the target position comprises the correction position or a first position; the first position is a position of the camera on the vehicle at the current time; a setting parameter of the camera at the correction position is deviated from a setting parameter of the camera at the first position, and the setting parameter comprises a direction angle and / or position information; Based on the plurality of images, a rotation matrix, a translation vector, a focal length, and coordinates of a central pixel point of the plurality of images of the camera are determined according to a preset calibration algorithm; An extrinsic parameter matrix of the camera is obtained according to the rotation matrix and the translation vector of the camera, and an intrinsic parameter matrix of the camera is obtained according to the focal length of the camera and the coordinates of the central pixel point of the plurality of images, so as to obtain parameter information of the camera; wherein the parameter information comprises intrinsic parameters and extrinsic parameters; when the target position is the correction position, the parameter information comprises standard intrinsic parameters and standard extrinsic parameters of the camera at the correction position; when the target position is the first position, the parameter information comprises actual intrinsic parameters and actual extrinsic parameters of the camera at the first position; The road image is corrected according to first parameter information and second parameter information of the camera, to obtain a corrected road image; wherein the first parameter information is parameter information of the camera at the correction position, and the second parameter information is parameter information of the camera at the first position; The corrected road image is identified to obtain information of a target object on the road where the vehicle travels.

2. The method of claim 1, wherein, The correction of the road image according to the first parameter information and the second parameter information of the camera to obtain the corrected road image comprises: Correction coefficients corresponding to the camera are determined according to the standard intrinsic parameters and the standard extrinsic parameters of the camera, and the actual intrinsic parameters and the actual extrinsic parameters of the camera; The road image is corrected according to the correction coefficients to obtain the corrected road image.

3. The method of claim 2, wherein, The correction coefficients are a ratio of a first value of the camera to a second value of the camera; the first value is a product of the standard intrinsic parameters and the standard extrinsic parameters of the camera; and the second value is a product of the actual intrinsic parameters and the actual extrinsic parameters of the camera.

4. The method according to any one of claims 1 to 3, characterized in that, The identification of the corrected road image to obtain the information of the target object on the road where the vehicle travels comprises: The corrected road image is input into a road information detection model to obtain the information of the target object, and the road information detection model has a function of identifying information of an object in an image.

5. An image recognition apparatus characterized by comprising: The application is applied to a vehicle which is provided with a camera, and the device comprises: A control unit is configured to control the camera to capture a road image of a road where the vehicle is located at a current time in response to a starting operation of the vehicle, and the road image is an image to be corrected; The control unit is further configured to, in response to the camera being at a target position on the vehicle, capture a plurality of images by the camera; the target position comprises the calibration position or a first position; the first position is a position of the camera on the vehicle at the current time; and a setting parameter of the camera at the calibration position is different from a setting parameter of the camera at the first position, the setting parameter comprising a direction angle and / or position information. The control unit is further configured to, based on the plurality of images, determine a rotation matrix and a translation vector of the camera, a focal length, and coordinates of a center pixel of the plurality of images according to a preset calibration algorithm; obtain an extrinsic matrix of the camera according to the rotation matrix and the translation vector of the camera, and obtain an intrinsic matrix of the camera according to the focal length and the coordinates of the center pixel of the plurality of images, to obtain parameter information of the camera; the parameter information comprises the intrinsic parameter and the extrinsic parameter; when the target position is the calibration position, the parameter information comprises a standard intrinsic parameter and a standard extrinsic parameter of the camera at the calibration position; and when the target position is the first position, the parameter information comprises an actual intrinsic parameter and an actual extrinsic parameter of the camera at the first position. A calibration unit is configured to calibrate the road image according to the first parameter information and the second parameter information of the camera, to obtain a calibrated road image; the first parameter information is parameter information of the camera at the calibration position, and the second parameter information is parameter information of the camera at the first position. An identification unit is configured to identify the calibrated road image, to obtain information of a target object on a road on which the vehicle travels.

6. An electronic device, comprising: An electronic device is applied to a vehicle, and comprises a memory and a processor; The memory and the processor are coupled; The memory is configured to store computer program code, the computer program code comprising computer instructions; When the processor executes the computer instructions, the electronic device performs the image recognition method according to any one of claims 1-4.

7. A computer-readable storage medium having stored therein instructions, the computer-readable storage medium comprising: When the instructions run on the electronic device, the electronic device performs the image recognition method according to any one of claims 1-4.

8. A vehicle characterized by comprising: The electronic device according to claim 6.

Images